Adoption guidelines for regulated and highly available workloads

HIPAA

If you're the account owner, you can enable your IBM Cloud® account to be HIPAA supported. For example, you might choose to enable the HIPAA Supported setting if you plan to include Protected Health Information (PHI) in HIPAA-enabled services.

The US Health Insurance Portability and Accountability Act (HIPAA) and the Health Information Technology for Economic and Clinical Health (HITECH) Act define standards for handling electronic healthcare transactions and information. If you or your company is a covered entity as defined by HIPAA, you must enable the HIPAA supported setting if you run sensitive workloads that are regulated under HIPAA and the HITECH Act. Learn more about IBM Cloud compliance in Compliance on the IBM Cloud.

When you configure your account to be HIPAA enabled, consider the following information:

• You can filter on HIPAA Enabled services in the catalog. Accounts that enable the HIPAA Supported setting still have access to the full catalog of services.
• You indicate to IBM that your account stores protected health information (PHI).
• You digitally accept the IBM Business Associate Addendum (BAA) for covered entities.

Enable this setting only if you or your company is a covered entity as defined by HIPAA. If you or your company is a business associate of a covered entity, contact IBM Cloud Sales to accept the applicable BAA. For more information about HIPAA definitions of covered entities and business associates, see the US Department of Health & Human Services website.

Locations

You can provision instances of the IBM Log Analysis service in any of the supported locations in the IBM Cloud. For more information, see Locations.

Per location (region), you can provision 1 or more logging instances.

• You can collect logs from custom applications and services that run in the IBM Cloud or outside, and forward them to any logging instance in your account.
• Only 1 instance in a location can be configured to collect logs automatically from IBM Cloud enabled services in that IBM Cloud location.
• Each location represents the geographic area where your IBM Log Analysis requests are handled and processed for that instance, and where data is resident.
• Each MZR location has three different data centers for redundancy. The data for each location is kept in the three data centers near that location. If all three data centers in a location fail, the IBM Log Analysis service for that location becomes unavailable.
• Each MZR configuration can accept a single data center failure.

When you choose the locations where you plan to provision IBM Log Analysis instances, check the regulatory and high availability (HA) specifications of each location.

For example, in Europe, only the Frankfurt region is EU-supported.

You can also check ensure zero downtime to learn more about the high availability and disaster recovery standards in IBM Cloud.

Platform logs

To enable automatic collection of IBM Cloud enabled services, you must configure 1 instance in a location with the platform logs flag. Learn more. When 1 instance is enabled to collect logs in a location, data from any instance of an enabled service in that location is collected automatically.

Define 1 instance per location with the platform logs flag enabled.

If you share staging, pre-production, and production services in the same IBM Cloud account, notice that users, that are granted access to view data in the logging instance with the platform logs flag in a location, can see data from any service instance provisioned in that location. To prevent users from viewing log data from all service's instances, consider moving from a single account model to an Enterprise account model.

If you cannot move to an enterprise account model, try reducing the number of users that are granted permissions to view the logs. In addition, you can also define exclusion rules to hide data from showing through the web UI. Exclusion rules stop logs from counting against your data usage quota and from being stored for search. Learn more.

Resource groups

Resource groups are a logical container for organizing your IAM-enabled resources.

• IAM-enabled services belong to a resource group. The IBM Log Analysis service is an IAM-enabled service.
• You assign a resource to its resource group when you create it from the catalog.
• You can't change the resource group assignment after you set it, which is why it's important to plan and set up your resource groups.
• If you add PII information in the resource group name, you might be disclosing sensitive data to others in the same account.

Use resource groups to organize your IBM Log Analysis instances for access control and billing purposes.

Account owners can add resources to any resource group. Other users must be granted access by using an IAM access policy to add resources to resource groups. For more information, see Best practices for organizing resources and assigning access.

When you define your resource groups, do not add sensitive information in the name.

Naming

If you add PII or other sensitive information in the name or the description of a service instance, you might be disclosing sensitive data to others in the same account.

When you define your logging instances names, do not add sensitive information in the name or in the description.

Service plan

The service plan that you choose for a logging instance determines the number of days that data is available for search. For more information, see Service plans.

Choose your plan based on the number of days that you need to be able to search data online through the web UI.

The HIPAA plan has a maximum of 25 users. If you need to grant permissions to more than 25 users, open a support case.

Tags

A tag is a label that you assign to a resource for easy filtering of resources in your resource list.

• You can use tags to organize your resources and easily find them later.
• You can also use tags to help you with identifying specific team usage or cost allocation when you view your exported usage report.

Tags are case-sensitive, and the maximum length of a tag is 128 characters.

• The characters that are permitted to name tags are A-Z, 0-9, spaces, underscore, hyphen, period, and colon.
• Colons turn the tag into a string where you can isolate two logical parts, like a key:value pair. You can't use a colon in a tag without creating this pairing.
• A comma separates tags and can't be used within the tag name itself.

If you add PII information in the name, you might be disclosing sensitive data to others in the same account.

When you define your tags, do not add sensitive information in the tag name.

Tags are visible to all members of an account.

To control tag visibility, circulate tagging guidelines, and let users know that tags are visible account-wide.

Access groups

You can assign permissions to work with the IBM Log Analysis service within the context of the service, a resource group, or an access group.

Use access groups to organize a set of users and service IDs into a single entity that makes it easy for you to manage IAM permissions.

You can create multiple access groups.

Define a minimum of 4 access groups:

Policies

A policy determines the full set of actions that a user or service ID can perform.

For each access group, define a policy for each resource group that defines the level of access to that resource group.

By default, the account owner is the only user in the account that can grant permissions to other users to manage and work with the IBM Log Analysis service.

To allow other users or service IDs in the account to manage the service and be able to grant permissions to work with the IBM Log Analysis service, define a policy for the IBM Log Analysis service with the platform role administrator. Grant this policy to the administrators access group.

For more information, see Granting permissions to a user to become an administrator of the service.

Access to resources within a resource group can be granted to all resources in a group, or only selected services within a group.

Access policies set a target, which is typically a service instance or all instances of a service in a resource group, and a role, which defines what type of access is allowed.

Roles define the actions that a user or serviceID can run. There are different types of roles in the IBM Cloud:

• Platform management roles define permissions to work with the service at the platform level, for example, some actions are assign user access for the service, create or delete service IDs, create instances, assign policies for your service to other users, and bind instances to applications. Learn more.
• Service access roles define permissions for calling the service's API. Learn more.

For each access group, define a policy for each resource group that specifies the permissions granted to work with the resource group, the permissions to manage instances of the IBM Log Analysis service in that resource group, and the permissions to run IBM Log Analysis tasks.

Every user that requires permissions to work with the IBM Log Analysis service in your account must be assigned a resource group policy that includes the permissions for the IBM Log Analysis service.

You can assign a single policy to the access group instead of assigning the same access multiple times per individual user or service ID.

Add users and service IDs to an access group. Grant permissions to these users and service IDs through the access group.

Learn more.

Backup the resource configurations of your logging instance

In the logging web UI, you can define custom views, dashboards, parsing templates, screens, and exclusion rules that you can use to view and analyze data.

To reuse resource definitions that you define in your logging instance, you can export these resources from an IBM Log Analysis instance as a JSON file. Then, you can import the definitions into other logging instances. For example, you can reuse your logging resources across different environments for your stage, pre-production, and production logging instances. Learn more.

Backup logging resource definitions into a version control system such as a git repository where you can control access to the archived files and manage versions.

Archive log data to a COS bucket

Enable archiving of your data from a logging instance to an IBM Cloud Object Storage (COS) bucket.

After you provision a logging instance, you can configure archiving to an IBM Cloud Object Storage (COS) bucket. You can create different types of buckets based on your requirements.

When you plan the bucket for a logging instance, consider the following information:

[1]: Data might sit untouched for long periods of time. For less active workloads, you can create buckets with different storage classes. For example, use the standard storage class for active workloads, where you need to access data at any time.

[2]: You can choose Immutable Object Storage to preserve electronic records and maintain data integrity. When you define a retention policy for a bucket, you are specifying that the data is stored in a WORM (Write-Once-Read-Many), non-erasable and non-rewritable manner. This policy is enforced until the end of a retention period and the removal of any legal holds. Notice that Immutable Object Storage is available in certain regions only.

[3]: You can use an expiration rule to delete objects automatically after a defined period from the object creation date.

[4]: You can define a retention policy to retain data for a long period of time and have the data available for downloads and queries. If you do not need to query the data, and you just need to keep it for compliance, look into archiving COS files from any of the storage tiers to a long-term offline archive location or use the online Cold Vault option.

[5]: COS provides several options to encrypt your data. By default, all objects that are stored in COS are encrypted by using randomly generated keys and an all-or-nothing-transform (AONT). With COS, you can also manage your keys manually by providing your own encryption keys - referred to as Server-Side Encryption with Customer-Provided Keys (SSE-C). Alternatively, you can choose to use the integration capabilities with IBM Cloud® Key Management Services like IBM® Key Protect and Hyper Protect Crypto Services.

[6]: Resiliency refers to the scope and scale of the geographic area across which your data is distributed. For example, you can choose cross region resiliency to spread your data across several geographical areas, or regional resiliency to spread data across a single region. Notice that a single data center distributes data across devices within a single site only.

Create a custom COS bucket with the storage features and the policies that you identify. Learn more.

Do not configure a long-term retention policy on a COS bucket if you need access to the data and query it with the Data Engine service.

Use the following table to help you identify the features that you should consider when you create a bucket:

Disaster recovery or compliance requirements

Requirement	Storage class: Standard[7]	Storage class: Vault[8]	Storage class: Cold Vault[9]	Storage class: Flex[10]
Type of workload	Continuous access	Data isn't accessed frequently	Accessed every 90 days or less	Dynamic workloads where access patterns are difficult to predict
Data resiliency	Cross region	Cross region	Cross region	Cross region
Retention policy
Expiration policy
Long-term retention policy	NO	NO	NO	NO

Backup or access to data beyond the service plan days requirements

Requirement	Storage class: Standard[7]	Storage class: Vault[8]	Storage class: Cold Vault[9]	Storage class: Flex[10]
Type of workload	Continuous access	Data isn't accessed frequently	Accessed every 90 days or less	Dynamic workloads where access patterns are difficult to predict
Data resiliency	Cross region or Regional	Cross region or Regional	Cross region or Regional	Cross region or Regional
Retention policy
Expiration policy
Long-term retention policy	NO	NO	NO	NO

[7]: Standard is used for active workloads. No charge incurs for data that is retrieved (other than the cost of the operational request itself and public outbound bandwidth).

[8]: Vault is used for cool workloads where data isn't accessed frequently, but a retrieval charge applies for reading data. The service includes a threshold for object size and storage period consistent with the intended use of this service for cooler, less-active data.

[9]: Cold Vault is used for cold workloads where data is primarily archived (accessed every 90 days or less) - a larger retrieval charge applies for reading data. The service includes a threshold for object size and storage period consistent with the intended use of this service: storing cold, inactive data.

[10]: Flex is used for dynamic workloads where access patterns are more difficult to predict. Depending on usage, if the cost of cooler storage that is combined with retrieval charges exceeds a cap value, then the storage charge increases and no any retrieval charges apply. If the data isn't accessed frequently, Flex storage can be more cost effective than Standard storage. If cooler usage patterns become more active, Flex storage is more cost effective than Vault or Cold Vault storage. No threshold object size or storage period applies to Flex buckets.

Objects that are subject to a bucket's Immutable Object Storage retention policy will have expiration actions deferred until the retention policy is no longer enforced.

Encrypt data with your own key

By default, all objects that are stored in COS are encrypted by using randomly generated keys and an all-or-nothing-transform (AONT). When you archive data into your COS instance, your objects are automatically encrypted with data encryption keys (DEKs). When you need to access a bucket, the service checks your user permissions and decrypts the objects within the bucket for you. This encryption model is called provider-managed encryption.

While this default encryption model provides at-rest security, some workloads need full control over the data encryption keys used.

COS provides several options to encrypt your data. This encryption model is called customer-managed encryption:

• You can manage your keys manually by providing your own encryption keys - referred to as Server-Side Encryption with Customer-Provided Keys (SSE-C).

• You can choose to use the integration capabilities with IBM Cloud® Key Management Services like IBM® Key Protect and Hyper Protect Crypto Services.

  You can use IBM® Key Protect to provision encrypted keys for apps across IBM Cloud services. As you manage the lifecycle of your keys, you can benefit from knowing that your keys are secured by FIPS 140-2 Level 3 certified cloud-based hardware security modules (HSMs) that protect against the theft of information. When you integrate COS with the Key Protect service, you can add envelope encryption to your DEKs. In Key Protect, you can provision highly secure root keys, which serve as primary keys that you control in the service. When you create a bucket, you can configure envelope encryption for the bucket at its creation. This added protection wraps (or encrypts) the DEKs associated with the bucket by using a root key that you manage in Key Protect. The practice, called key wrapping, uses multiple AES algorithms to protect the privacy and the integrity of your DEKs, so only you control access to their associated data.

  Hyper Protect Crypto Services is a single-tenant, dedicated HSM that is controlled by you. The service is built on FIPS 140-2 Level 4-certified hardware, the highest offered by any cloud provider in the industry.

If you need to use your own key to encrypt the data at-rest in a bucket, use the following table to help you identify the features that you should consider when you create a bucket:

Use your own encryption key

Requirement	Storage class: Standard	Storage class: Vault	Storage class: Cold Vault	Storage class: Flex
Type of workload	Continuous access	Data isn't accessed frequently	Accessed every 90 days or less	Dynamic workloads where access patterns are difficult to predict
Data resiliency	Regional	Regional	Regional	Regional
Retention policy [15]	NO	NO	NO	NO
Expiration policy
Long-term retention policy	NO	NO	NO	NO

[15]: Currently, you cannot use your own key with a bucket that has a retention policy configured.

Use the customer-managed encryption model to manage and control the key that is used to encrypt data at-rest.

After you designate a root key in Key Protect and grant access between your services, you can enable envelope encryption for a specified storage bucket by using the IBM Cloud Object Storage GUI.

To enable encryption with your custom key by using the Key Protect service, create a root key and create an authorization between your COS instance and the Key Protect instance. Notice that the COS bucket and the Key Protect instance need to be available in the same region.

Naming

If you add PII information in the name of a bucket, you might be disclosing sensitive data to others in the same account.

When you define your bucket name, do not add sensitive information in the name or in the description.

IAM resources to configure archiving

To configure archiving, you need the following IAM policies:

• An IAM policy with minimum platform role Viewer and service role Manager for the IBM Log Analysis service: This policy is required to access the logging web UI, and configure archiving of the logging instance.
• An IAM policy with minimum platform role Administrator and service role Manager for the COS service: The administrator role is required to define the service ID, with its associated API key, that is used by the IBM Log Analysis instance to authenticate and access the IBM Cloud Object Storage instance. The manager role is required to create and configure a bucket.

You also need a service ID. A service ID identifies a service similar to how a user ID identifies a user. Service IDs are not tied to a specific user. If the user that creates the service ID leaves your organization and is deleted from the account, the service ID remains. The IBM Log Analysis service uses an API key that is associated to a service ID that you define on your COS instance to authenticate and write files to the COS bucket.

Create a service ID for for the COS instance with writer permissions. Restrict access to the service ID so that the API key that is associated to it can only write to the bucket that you configure in logging for archiving.

If you add PII information in the name or the description of the service ID, you might be disclosing sensitive data to others in the same account.

When you define your service ID name, do not add sensitive information in the name or in the description.

You might have a requirement to rotate API keys regularly or your API key might be compromised.

Rotate the API key that is associated with your service ID regularly to prevent any security breaches caused by leaked keys or to comply with security guidelines.

IAM policies to control access to the archived files

In COS, you can define policies to control the permissions that are granted to service IDs and users to read, write, update object properties, and delete objects. Learn more.

You will want to restrict user access to archived files in the bucket.

EU supported account

When you archive logs from the Frankfurt logging instance to an IBM Cloud Object Storage (COS) bucket, consider the following information:

• When you provision an instance of the COS service, this instance is a global one in your account. It is not region bound.

• You must configure a bucket that complies with the EU-Supported and GDPR regulations. For the list of COS EU-supported endpoints, see EU-supported endpoints.

  For example, consider the following scenarios:

  • For a bucket with regional resiliency, you can create the bucket in the EU-DE location to keep the data in Frankfurt.

  • For a bucket with cross region resiliency, you can create the bucket in the eu-geo location. Data is kept within the EU geography across datacenters that are located in Milan, Amsterdam, and Frankfurt.

• You must restrict user access to manage archived log files in these buckets.

• Users are responsible for downloading files to EU-supported locations.

Monitor archiving through Activity Tracker

You can use the IBM Cloud® Activity Tracker service to monitor the activity of your IBM Cloud account. You can use this service to investigate for abnormal activity and critical actions, and comply with regulatory audit requirements. In addition, you can be alerted on actions as they happen. The events that are collected comply with the Cloud Auditing Data Federation (CADF) standard.

In COS, you can track management and data events.

• By default, these events are not enabled.
• Management events report actions that change the state of resource configurations, bucket properties, and object properties.
• Data events report actions on buckets, objects, and multi-part objects such as creation, deletion, and access.
• You must configure each bucket to enable management events, or management and data events. Notice that you cannot enable data events only for a bucket.

For more information, see Activity Tracker events.

Enable collection of COS management and data events on the bucket that you use to archive data from a logging instance. Use these events to monitor activity in your COS bucket.

In IBM Cloud® Activity Tracker, you can define views, dashboard, and screens to monitor COS management and data events. You can also configure alerts on views to notify you when a specific condition occurs. On a view, you can configure an email alert, a Slack alert, a PagerDuty alert, or any combination of alerts. For more information, see Creating custom views and Managing alerts.

For example, you can define a view that reports when an object is archived in a bucket. On that view, you can configure an email alert to notify you when an object has been archived. You might have compliance requirements that require you to control who accesses data that is archived. You can define a view that reports access to a bucket, and define an alert to notify you when that happens.

Define views, dashboard, screens, and alerts in IBM Cloud® Activity Tracker to investigate for abnormal activity and critical actions in your COS bucket, and to comply with regulatory audit requirements.

Archived file

Although you define policies to manage your COS objects (logging archived files) at the bucket level, there might be situations where you need to hold on to specific files for longer for compliance and auditing purposes. In COS, you can set different attributes per object that allow you to meet this requirements.

• You can define a legal hold flag to an archive file. Legal holds can be applied to objects during initial uploads or after an object is written. Learn more.
• You can define an indefinite retention flag to store the object indefinitely until a new retention period is applied. Learn more.

Add a legal hold flag or an indefinite retention flag to individual archived files if you need to keep a file longer than the default retention period specified at the bucket level.