IBM Cloud Docs
Adoption guidelines for regulated and highly available workloads

Adoption guidelines for regulated and highly available workloads

For regulated and highly available workloads, consider the following adoption guidelines when using the IBM Cloud® Activity Tracker (AT) service:

As of 28 March 2024 the IBM Log Analysis and IBM Cloud Activity Tracker services are deprecated and will no longer be supported as of 30 March 2025. Customers will need to migrate to IBM Cloud Logs, which replaces these two services, prior to 30 March 2025. For information about IBM Cloud Logs, see the IBM Cloud Logs documentation.

Define resources naming standards for compliance

When you create resources in the IBM Cloud, you can choose how to name them, what information to include in their description fields, which tags to use to group them, associate metadata, and more.

Define naming standards that do not include PII and other sensitive information for all resources that are created in the IBM Cloud.

Define the account management strategy

In IBM Cloud, you can have 1 or more stand-alone accounts. You can manage each account individually or within an enterprise by configuring a multitiered hierarchy of accounts.

Within an enterprise account, you create a multitiered hierarchy of accounts, with billing and payments for all accounts managed at the enterprise level. Learn more.

  • The enterprise account serves as the parent account to all other accounts in the enterprise.
  • Users and access management is isolated between the enterprise and its child accounts. No access is automatically inherited between the two types of accounts.
  • Resources and services within an enterprise function the same as in stand-alone accounts. Each account in an enterprise can contain resources in resource groups.

An enterprise can contain up to 5 tiers of accounts and account groups. In its most basic form, an enterprise has two tiers: the enterprise account, and a single child account.

The following table highlights some of the key features per account management strategy:

Types of accounts
Feature Stand-alone account management Enterprise account management
Multitiered hierarchy of accounts Checkmark icon
Billing and payments managed from 1 account Checkmark icon Checkmark icon
Isolation of users and access management per account Checkmark icon Checkmark icon
Isolation of resources and services per account Checkmark icon Checkmark icon
Isolation of account settings Checkmark icon Checkmark icon
IAM enabled Checkmark icon Checkmark icon

In stand-alone accounts, you control access to resources by grouping them in resource groups, and configuring IAM policies that you assign to users and service IDs directly or through access groups. These policies define the level of access to work with services in the account. For example, you might have a stand-alone account where you run your development, pre-production, and production services and applications.

By using an enterprise management account strategy, you achieve greater isolation of resources. When you configure a multitiered hierarchy of accounts, you get the flexibility to separate your development environments into separate tiers, or isolate by Line of Business (LoB) applications and services, or a combination of both. Each account retains the features of a stand-alone account, and you can still manage billing and payments from the enterprise account.

In addition to these benefits, when you look into auditing and how it fits with any of these management strategies, consider the following that will be covered later in more detail:

  • Auditing events are collected and available per location (region) through 1 single auditing instance. You do not have the ability to split events from different services running in the same region to multiple instances. If you run your development, test, and production services in a stand-alone account, all those events will be available through the same auditing instance, and any user with permissions to view events in that instance will be able to see everything.
  • The auditing instance in Frankfurt collects events from global services such as IAM.

Define an enterprise account management strategy to add an additional layer of isolation to resources.

Configure account settings for compliance

Across every industry, organizations require tight controls and visibility into where their data is stored and processed.

Indicate to IBM your compliance requirements by enabling your IBM Cloud account or IBM Cloud Entreprise account as HIPAA or EU supported.

In the IBM Cloud, you can configure your account for EU support and for HIPAA support:

  • You might choose to enable the EU Supported setting, for example, if you use resources to process personal data for European citizens. For more information, see Set the EU-Supported flag on in your account.
  • You might choose to enable the HIPAA Supported setting if you plan to include Protected Health Information (PHI) in HIPAA-enabled services. For more information, see Set the HIPAA flag on in your account.

Only the account owner can enable the account to be EU supported and HIPAA supported.

HIPAA

If you're the account owner, you can enable your IBM Cloud® account to be HIPAA supported. For example, you might choose to enable the HIPAA Supported setting if you plan to include Protected Health Information (PHI) in HIPAA-enabled services.

The US Health Insurance Portability and Accountability Act (HIPAA) and the Health Information Technology for Economic and Clinical Health (HITECH) Act define standards for handling electronic healthcare transactions and information. If you or your company is a covered entity as defined by HIPAA, you must enable the HIPAA supported setting if you run sensitive workloads that are regulated under HIPAA and the HITECH Act. Learn more about IBM Cloud compliance in Compliance on the IBM Cloud.

When you configure your account to be HIPAA enabled:

  • You can filter on HIPAA Enabled services in the catalog. Accounts that enable the HIPAA Supported setting still have access to the full catalog of services.
  • You indicate to IBM that your account stores protected health information (PHI).
  • You digitally accept the IBM Business Associate Addendum (BAA) for covered entities.

Enable this setting only if you or your company is a covered entity as defined by HIPAA. If you or your company is a business associate of a covered entity, contact IBM Cloud Sales to accept the applicable BAA. For more information about HIPAA definitions of covered entities and business associates, see the US Department of Health & Human Services website.

Define the auditing instances strategy

IBM Cloud Activity Tracker collects and aggregates events in one centralized system.

  • Data is hosted on the IBM Cloud.
  • Data is collocated in the location where an IBM Cloud Activity Tracker instance is provisioned, except for events that report on global actions.

Locations

You can provision instances of the IBM Cloud Activity Tracker service in any of the supported locations in the IBM Cloud. For more information, see Locations.

Per location (region), you can provision 1 auditing instance.

  • Events from enabled-AT services in the IBM Cloud are collected automatically.
  • Each location represents the geographic area where your IBM Cloud Activity Tracker requests are handled and processed for that instance, and where data is resident.
  • Each MZR location has three different data centers for redundancy. The data for each location is kept in the three data centers near that location. If all three data centers in a location fail, the IBM Cloud Activity Tracker service for that location becomes unavailable.
  • Each MZR configuration can accept a single data center failure.

When you choose the locations where you plan to provision IBM Cloud Activity Tracker instances, check the regulatory and high availability (HA) specifications of each location.

For example, in Europe, only the Frankfurt region is EU-supported.

You can also check ensure zero downtime to learn more about the high availability and disaster recovery standards in IBM Cloud.

Resource groups

Resource groups are a logical container for organizing your IAM-enabled resources.

  • IAM-enabled services belong to a resource group. The IBM Cloud Activity Tracker service is an IAM-enabled service.
  • You assign a resource to its resource group when you create it from the catalog.
  • You can't change the resource group assignment after you set it, which is why it's important to plan your resource groups.
  • If you add PII information in the resource group name, you might be disclosing sensitive data to others in the same account.

Use a resource group, that only includes Activity Tracker instances, to organize your IBM Cloud Activity Tracker instances for access control and billing purposes.

Account owners can add resources to any resource group. Other users must be granted access by using an IAM access policy to add resources to resource groups. For more information, see Best practices for organizing resources and assigning access.

When you define your resource groups, do not add sensitive information in the name.

Naming

If you add PII or other sensitive information in the name or the description of a service instance, you might be disclosing sensitive data to others in the same account.

When you define your auditing instances names, do not add sensitive information in the name or in the description.

Service plan

The service plan that you choose determines the number of days that data is available for search. For more information, see Service plans.

Choose your plan based on the number of days that you need to be able to search data online through the web UI.

The HIPAA plan has a maximum of 25 users. If you need to grant permissions to more than 25 users, open a support ticket.

Tags

A tag is a label that you assign to a resource for easy filtering of resources in your resource list.

  • You can use tags to organize your resources and easily find them later.
  • You can also use tags to help you with identifying specific team usage or cost allocation when you view your exported usage report.

Use tags to organize your resources and track usage costs.

Tags are case-sensitive, and the maximum length of a tag is 128 characters.

  • The characters that are permitted to name tags are A-Z, 0-9, spaces, underscore, hyphen, period, and colon.
  • Colons turn the tag into a string where you can isolate two logical parts, like a key:value pair. You can't use a colon in a tag without creating this pairing.
  • A comma separates tags and can't be used within the tag name itself.

If you add PII information in the name, you might be disclosing sensitive data to others in the same account.

When you define your tags, do not add sensitive information in the tag name.

Tags are visible to all members of an account.

To control tag visibility, circulate tagging guidelines, and let users know that tags are visible account-wide.

Define the IAM strategy

Use IBM Cloud® Identity and Access Management (IAM) to securely authenticate users and service IDs, and to control access to all cloud resources and data consistently in the IBM Cloud.

If you add PII information in the name or description of IAM resources, you might be disclosing sensitive data to others in the same account.

When you define your IAM resources, do not add sensitive information in their names and descriptions.

Access groups

You can assign permissions to work with the IBM Cloud Activity Tracker service within the context of the service, a resource group, or an access group.

Use access groups to organize a set of users and service IDs into a single entity that makes it easy for you to manage IAM permissions.

You can create multiple access groups.

Define a minimum of 4 access groups:

List of access groups
Access group Description
Administrators Users in this group should have permissions to fully manage the service and grant other users in the account permissions to work with the service in the IBM Cloud.
Managers Users in this group should have permissions to fully manage the service in the IBM Cloud.
Advanced service users Users in this group should have permissions to run advanced service tasks.
Users Users in this group should have permissions to run basic tasks.

Policies

A policy determines the full set of actions that a user or service ID can perform.

For each access group, define a policy for each resource group that defines the level of access to that resource group.

By default, the account owner is the only user in the account that can grant permissions to other users to manage and work with the IBM Cloud Activity Tracker service.

To allow other users or service IDs in the account to manage the service and be able to grant permissions to work with the IBM Cloud Activity Tracker service, define a policy for the IBM Cloud Activity Tracker service with the platform role administrator. Grant this policy to the administrators access group.

Access to resources within a resource group can be granted to all resources in a group, or only selected services within a group.

Access policies set a target, which is typically a service instance or all instances of a service in a resource group, and a role, which defines what type of access is allowed.

Roles define the actions that a user or serviceID can run. There are different types of roles in the IBM Cloud:

  • Platform management roles define permissions to work with the service at the platform level, for example, some actions are assign user access for the service, create or delete service IDs, create instances, assign policies for your service to other users, and bind instances to applications. Learn more.
  • Service access roles define permissions for calling the service's API. Learn more.

For each access group, define a policy for each resource group that specifies the permissions granted to work with the resource group, the permissions to manage instances of the IBM Cloud Activity Tracker service in that resource group, and the permissions to run IBM Cloud Activity Tracker tasks.

Roles per access group policy
Access group Policy
Administrators Grant a policy for the resource group with resource group access administrator. Select the IBM Cloud Activity Tracker service, and select the platform role administrator, and the service role manager.
Managers Grant a policy for the resource group with platform role editor. Select the IBM Cloud Activity Tracker service, and select the platform role editor, and the service role manager.
Advanced service users Grant a policy for the resource group with platform role viewer. Select the IBM Cloud Activity Tracker service, and select the platform role viewer, and the service role standard member.
Users Grant a policy for the resource group with platform role viewer. Select the IBM Cloud Activity Tracker service, and select the platform role viewer, and the service role reader.

Every user that requires permissions to work with the IBM Cloud Activity Tracker service in your account must be assigned a resource group policy that includes the permissions for the IBM Cloud Activity Tracker service.

You can assign a single policy to the access group instead of assigning the same access multiple times per individual user or service ID.

Add users and service IDs to an access group. Grant permissions to these users and service IDs through the access group.

For more information, see Granting administration permissions to a user or service ID.

Configure the account settings for authentication into your account

Multi-factor authentication (MFA) adds an extra layer of security to your account by requiring all users to authenticate by using an additional authentication method beyond an ID and password. This is also commonly known as two-factor authentication (2FA).

The IBM Cloud account owner or administrator for the billing service can choose to require multi-factor authentication (MFA) for every user in the account or just users with non-federated IDs who do not use SSO. All users with an IBMid use a time-based one-time passcode (TOTP) MFA method, and any users with a different type of ID must be enabled to use the time-based one-time passcode authentication (TOTP), security questions, or an external authentication method separately. Learn more.

You can also configure MFA options such as security questions, using a time-based one-time passcode, and using an external authentication method. These types of MFA options are specific per account and are available only with former classic infrastructure accounts.

Enable multi-factor authentication (MFA) in your IBM Cloud account for all users.

Define the notification strategy

In an auditing instance, you define views to analyze the data. Then, you can configure 1 or more alerts per view to notify of an abnormal situation. Learn more.

You can choose to be notified by using an absence alert that is triggered when no data is available.

Define an absence alert to be notified when inactivity in an application or service is identified. Notice that absence alerts require data in the view within the past 24 hours for the alert to be active.

You can choose to be notified by using a presence alert that is triggered when more log lines than expected are present.

Define a presence alert to be notified of exceptional situations in your applications and services that require immediate attention.

You can configure multiple notification channels. Valid channels are: email, Slack, PagerDuty, Webhook

Guidance for notification channel
Channel Guidance
email Email is a traditional communication method that you can use to notify 1 or more users. This notification channel requires users to be monitoring proactively their emails to detect an alert. Define an email alert to verify that the alert on a view is working, and to inform users of situations they may have requested information.
Slack Slack is a collaborative tool that you can use to communicate and share information across 1 or more teams. Define a Slack alert to inform about routine scenarios that you might want to monitor.
PagerDuty PagerDuty is an incident management tool that you can use to automate incident resolution and escalation, define on-call schedules and more. Define a PagerDuty alert to be notified immediately so that you can take action promptly.
Webhook A webhook is another method that you can configure to provide other applications information. Define a webhook alert if you have a third party tool that you can configure with an auditing instance using a webhook, and where you plan to manage notifications.

Configure PagerDuty alerts to be notified immediately so that you can take action promptly on problems and automate their resolution. In addition, configure Slack alerts to share alerts and information.

You can also define a preset. A preset is an alert template that you can attach to any number of views.

To reuse an alert configuration with different views and enforce notification channels across users that analyze data through that instance, configure alert presets.

When you send a notification, you can include log data as part of the notification.

Customize the data that is included in a notification for situations where the receiver of the notification does not have access to the log data.

Define the archive strategy

You might have different requirements that require archiving your data:

  • Backup requirements to keep data for a period of time
  • Data access requirements so that you can query the data after it is not available for search through the web UI
  • Disaster recovery requirements
  • Compliance requirements

IBM Cloud Activity Tracker does not backup your data.

There are 2 types of data that you should consider archiving:

  • Log data

    By default, archiving of log data is not enabled for any auditing instance.

    When you enable archiving of your log data, you are responsible for checking that your archived files are not corrupted, and for the maintenance of your archived files.

  • Web UI resource definitions such as parsing templates, exclusion rules, views, screens, and dashboards.

Archive your resource definitions and your log data.

Backup the resource configurations of your auditing instance

In the UI, you can define custom views, dashboards, parsing templates, screens, and exclusion rules that you can use to view and analyze data.

To reuse resource definitions that you define in your auditing instance, you can export these resources from an IBM Cloud Activity Tracker instance as a JSON file. Then, you can import the definitions into other auditing instances. For example, you can reuse your resources across different environments for your stage, pre-production, and production instances. Learn more.

Backup resource definitions into a version control system such as a git repository where you can control access to the archived files and manage versions.

Archive log data to a COS bucket

Enable archiving of your data from an auditing instance to an IBM Cloud Object Storage (COS) bucket.

After you provision an auditing instance, you can configure archiving to an IBM Cloud Object Storage (COS) bucket. You can create different types of buckets based on your requirements.

When you plan the bucket for an auditing instance, consider the following information:

COS bucket requirements
Requirement Question to answer Information
Type of workload [1] How often do you need to access the data? Information about storage classes
Retention policy [2] Do you need to protect data from being deleted? Information about Immutable Object Storage
Expiration policy [3] Do you need automatic deletion of files? Information about expiration rules
Long-term retention policy [4] Do you need to keep data for a long period of time? Information about archiving COS files for long term storage
Encryption of data at-rest with my own key [5] Can I use my own key to encrypt data at-rest? Information about encryption of data at-rest
Data resiliency [6] Do you need to store the data in a specific geographical location? Information about resiliency

[1]: Data might sit untouched for long periods of time. For less active workloads, you can create buckets with different storage classes. For example, use the standard storage class for active workloads, where you need to access data at any time.

[2]: You can choose Immutable Object Storage to preserve electronic records and maintain data integrity. When you define a retention policy for a bucket, you are specifying that the data is stored in a WORM (Write-Once-Read-Many), non-erasable and non-rewritable manner. This policy is enforced until the end of a retention period and the removal of any legal holds. Notice that Immutable Object Storage is available in certain regions only.

[3]: You can use an expiration rule to delete objects automatically after a defined period from the object creation date.

[4]: You can define a retention policy to retain data for a long period of time and have the data available for downloads and queries. If you do not need to query the data, and you just need to keep it for compliance, look into archiving COS files from any of the storage tiers to a long-term offline archive location or use the online Cold Vault option.

[5]: COS provides several options to encrypt your data. By default, all objects that are stored in COS are encrypted by using randomly generated keys and an all-or-nothing-transform (AONT). With COS, you can also manage your keys manually by providing your own encryption keys - referred to as Server-Side Encryption with Customer-Provided Keys (SSE-C). Alternatively, you can choose to use the integration capabilities with IBM Cloud® Key Management Services like IBM® Key Protect and Hyper Protect Crypto Services.

[6]: Resiliency refers to the scope and scale of the geographic area across which your data is distributed. For example, you can choose cross region resiliency to spread your data across several geographical areas, or regional resiliency to spread data across a single region. Notice that a single data center distributes data across devices within a single site only.

Create a custom COS bucket with the storage features and the policies that you identify. Learn more.

Do not configure a long-term retention policy on a COS bucket if you need access to the data and need to query it with the Data Engine service.

Use the following table to help you identify the features that you should consider when you create a bucket:

Disaster recovery or compliance requirements
Requirement Storage class: Standard[7] Storage class: Vault[8] Storage class: Cold Vault[9] Storage class: Flex[10]
Type of workload Continuous access Data isn't accessed frequently Accessed every 90 days or less Dynamic workloads where access patterns are difficult to predict
Data resiliency Cross region Cross region Cross region Cross region
Retention policy Checkmark icon Checkmark icon Checkmark icon Checkmark icon
Expiration policy Checkmark icon Checkmark icon Checkmark icon Checkmark icon
Long-term retention policy NO NO NO NO
Backup or access to data beyond the service plan days requirements
Requirement Storage class: Standard[7] Storage class: Vault[8] Storage class: Cold Vault[9] Storage class: Flex[10]
Type of workload Continuous access Data isn't accessed frequently Accessed every 90 days or less Dynamic workloads where access patterns are difficult to predict
Data resiliency Cross region or Regional Cross region or Regional Cross region or Regional Cross region or Regional
Retention policy Checkmark icon Checkmark icon Checkmark icon Checkmark icon
Expiration policy Checkmark icon Checkmark icon Checkmark icon Checkmark icon
Long-term retention policy NO NO NO NO

[7]: Standard is used for active workloads. No charge incurs for data that is retrieved (other than the cost of the operational request itself and public outbound bandwidth).

[8]: Vault is used for cool workloads where data isn't accessed frequently, but a retrieval charge applies for reading data. The service includes a threshold for object size and storage period consistent with the intended use of this service for cooler, less-active data.

[9]: Cold Vault is used for cold workloads where data is primarily archived (accessed every 90 days or less) - a larger retrieval charge applies for reading data. The service includes a threshold for object size and storage period consistent with the intended use of this service: storing cold, inactive data.

[10]: Flex is used for dynamic workloads where access patterns are more difficult to predict.

Objects that are subject to a bucket's Immutable Object Storage retention policy will have expiration actions deferred until the retention policy is no longer enforced.

Encrypt data with your own key

By default, all objects that are stored in COS are encrypted by using randomly generated keys and an all-or-nothing-transform (AONT). When you archive data into your COS instance, your objects are automatically encrypted with data encryption keys (DEKs). When you need to access a bucket, the service checks your user permissions and decrypts the objects within the bucket for you. This encryption model is called provider-managed encryption.

While this default encryption model provides at-rest security, some workloads need full control over the data encryption keys used.

COS provides several options to encrypt your data. This encryption model is called customer-managed encryption:

  • You can manage your keys manually by providing your own encryption keys - referred to as Server-Side Encryption with Customer-Provided Keys (SSE-C).

  • You can choose to use the integration capabilities with IBM Cloud® Key Management Services like IBM® Key Protect and Hyper Protect Crypto Services.

    You can use IBM® Key Protect to provision encrypted keys for apps across IBM Cloud services. As you manage the lifecycle of your keys, you can benefit from knowing that your keys are secured by FIPS 140-2 Level 3 certified cloud-based hardware security modules (HSMs) that protect against the theft of information. When you integrate COS with the Key Protect service, you can add envelope encryption to your DEKs. In Key Protect, you can provision highly secure root keys, which serve as primary keys that you control in the service. When you create a bucket, you can configure envelope encryption for the bucket at its creation. This added protection wraps (or encrypts) the DEKs associated with the bucket by using a root key that you manage in Key Protect. The practice, called key wrapping, uses multiple AES algorithms to protect the privacy and the integrity of your DEKs, so only you control access to their associated data.

    Hyper Protect Crypto Services is a single-tenant, dedicated HSM that is controlled by you. The service is built on FIPS 140-2 Level 4-certified hardware, the highest offered by any cloud provider in the industry.

If you need to use your own key to encrypt the data at-rest in a bucket, use the following table to help you identify the features that you should consider when you create a bucket:

Use your own encryption key
Requirement Storage class: Standard Storage class: Vault Storage class: Cold Vault Storage class: Flex
Type of workload Continuous access Data isn't accessed frequently Accessed every 90 days or less Dynamic workloads where access patterns are difficult to predict
Data resiliency Regional Regional Regional Regional
Retention policy [15] NO NO NO NO
Expiration policy Checkmark icon Checkmark icon Checkmark icon Checkmark icon
Long-term retention policy NO NO NO NO

[15]: Currently, you cannot use your own key with a bucket that has a retention policy configured.

Use the customer-managed encryption model to manage and control the key that is used to encrypt data at-rest.

After you designate a root key in Key Protect and grant access between your services, you can enable envelope encryption for a specified storage bucket by using the IBM Cloud Object Storage GUI.

To enable encryption with your custom key by using the Key Protect service, create a root key and create an authorization between your COS instance and the Key Protect instance. Notice that the COS bucket and the Key Protect instance need to be available in the same region.

Naming

If you add PII information in the name of a bucket, you might be disclosing sensitive data to others in the same account.

When you define your bucket name, do not add sensitive information in the name or in the description.

IAM resources to configure archiving

To configure archiving, you need the following IAM policies:

  • An IAM policy with minimum platform role Viewer and service role Manager for the IBM Cloud Activity Tracker service: This policy is required to access the UI, and configure archiving of the auditing instance.
  • An IAM policy with minimum platform role Administrator and service role Manager for the COS service: The administrator role is required to define the service ID, with its associated API key, that is used by the IBM Cloud Activity Tracker instance to authenticate and access the IBM Cloud Object Storage instance. The manager role is required to create and configure a bucket.

You also need a service ID. A service ID identifies a service similar to how a user ID identifies a user. Service IDs are not tied to a specific user. If the user that creates the service ID leaves your organization and is deleted from the account, the service ID remains. The IBM Cloud Activity Tracker service uses an API key that is associated to a service ID that you define on your COS instance to authenticate and write files to the COS bucket.

Create a service ID for for the COS instance with writer permissions. Restrict access to the service ID so that the API key that is associated to it can only write to the bucket that you configure for archiving.

If you add PII information in the name or the description of the service ID, you might be disclosing sensitive data to others in the same account.

When you define your service ID name, do not add sensitive information in the name or in the description.

You might have a requirement to rotate API keys regularly or your API key might be compromised.

Rotate the API key that is associated with your service ID regularly to prevent any security breaches caused by leaked keys or to comply with security guidelines.

IAM policies to control access to the archived files

In COS, you can define policies to control the permissions that are granted to service IDs and users to read, write, update object properties, and delete objects. Learn more.

Restrict user access to archived files in the bucket.

EU supported account

When you archive logs from the Frankfurt auditing instance to an IBM Cloud Object Storage (COS) bucket, consider the following information:

  • When you provision an instance of the COS service, this instance is a global one in your account. It is not region bound.

  • You must configure a bucket that complies with the EU-supported and GDPR regulations. For the list of COS EU-supported endpoints, see EU-supported endpoints.

    For example, consider the following scenarios:

    • For a bucket with regional resiliency, you can create the bucket in the EU-DE location to keep the data in Frankfurt.

    • For a bucket with cross region resiliency, you can create the bucket in the eu-geo location. Data is kept within the EU geography across datacenters that are located in Milan, Amsterdam, and Frankfurt.

  • You must restrict user access to manage archived log files in these buckets.

  • Users are responsible for downloading files to EU-supported locations.

Monitor archiving through Activity Tracker

You can use the IBM Cloud® Activity Tracker service to monitor the activity of your IBM Cloud account. You can use this service to investigate for abnormal activity and critical actions, and comply with regulatory audit requirements. In addition, you can be alerted on actions as they happen. The events that are collected comply with the Cloud Auditing Data Federation (CADF) standard.

In COS, you can track management and data events.

  • By default, these events are not enabled.
  • Management events report actions that change the state of resource configurations, bucket properties, and object properties.
  • Data events report actions on buckets, objects, and multi-part objects such as creation, deletion, and access.
  • You must configure each bucket to enable management events, or management and data events. Notice that you cannot enable data events only for a bucket. For more information, see Activity Tracker events.

Enable collection of COS management and data events on the bucket that you use to archive data from an auditing instance. Use these events to monitor activity in your COS bucket.

In IBM Cloud® Activity Tracker, you can define views, dashboard, and screens to monitor COS management and data events. You can also configure alerts on views to notify you when a specific condition occurs. On a view, you can configure an email alert, a Slack alert, a PagerDuty alert, or any combination of these. For more information, see Creating custom views and Managing alerts.

For example, you can define a view that reports when an object is archived in a bucket. On that view, you can configure an email alert to notify you when an object has been archived. You might have compliance requirements that require you to control who accesses data that is archived. You can define a view that reports access to a bucket, and define an alert to notify you when that happens.

Define views, dashboard, screens, and alerts in IBM Cloud® Activity Tracker to investigate for abnormal activity and critical actions in your COS bucket, and to comply with regulatory audit requirements.

Archived file

Although you define policies to manage your COS objects (IBM Cloud Activity Tracker archived files) at the bucket level, there might be situations where you need to hold on to specific files for longer for compliance and auditing purposes. In COS, you can set different attributes per object that allow you to meet this requirements.

  • You can define a legal hold flag to an archive file. Legal holds can be applied to objects during initial uploads or after an object is written. Learn more.
  • You can define an indefinite retention flag to store the object indefinitely until a new retention period is applied. Learn more.

Add a legal hold flag or an indefinite retention flag to individual archived files if you need to keep a file longer than the default retention period specified at the bucket level.

Define the strategy to query archived data

You can download data locally, and then use your own tools to query the data. When you download data for analysis, you are responsible for ensuring that your users comply with the regulations and compliance requirements that may be required for your organization. For example, if you must comply with GDPR, you need to control that users are following the download guidelines per GDPR rules.

You can use the Data Engine service to query IBM Cloud Activity Tracker archived files that are stored in a COS bucket in your account. You can run queries from the IBM Cloud UI, or programmatically.

Avoid downloading data locally. Use the Data Engine service to query archived data.

The Data Engine service provides a server-less, no-ETL solution to easily query data stored in Object Storage. SQL Query uses Apache Spark SQL as its underlying query engine. You can use the Data Engine to run SQL queries (that is, SELECT statements) to analyze, transform structured and semi-structured data, or clean up rectangular data. You cannot run actions such as CREATE, DELETE, INSERT, and UPDATE.

The Data Engine service can process input data that is read from CSV, JSON, ORC, Parquet, or AVRO files. Archived files from an IBM Cloud Activity Tracker instance contain data in JSON format. When you use the Data Engine service, each query result can be written to a CSV, JSON, ORC, PARQUET, or AVRO file in an Object Storage instance of your choice.

When you query an IBM Cloud Activity Tracker archive file, you must convert the JSON formatted file into PARQUET format to be able to query the contents successfully.

Use the Data Engine user interface (UI) to develop and test your queries, and the SQL Query REST API to automate them.

If you plan to use the Data Engine service, and you require HIPAA compliance, create a bucket for your archives that uses a custom key to encrypt the data.