Server-Side Encryption with Hyper Protect Crypto Services

You can use Hyper Protect Crypto Services to create, add, and manage keys, which you can then associate with your instance of IBM® Cloud Object Storage to encrypt buckets.

This feature is not currently supported in Object Storage for Satellite. Learn more.

Before you begin

Before you plan on using Hyper Protect Crypto Services with Cloud Object Storage buckets, you need:

An IBM Cloud™ Platform account
An instance of IBM Cloud Object Storage with a standard pricing plan.

You will need to ensure that a service instance is created by using the IBM Cloud catalog and appropriate permissions are granted. This section outlines step-by-step instructions to help you get started.

Provisioning an instance of Hyper Protect Crypto Services

Refer to the service-specific product pages for instructions on how to provision and setup appropriate service instances.

Getting started with Hyper Protect Crypto Services

Once you have an instance of Hyper Protect Crypto Services, you need to create a root key and note the CRN (Cloud Resource Name) of that key. The CRN is sent in a header during bucket creation.

Before creating the bucket for use with Hyper Protect Crypto Services, review the relevant guidance around availability and disaster recovery.

Create or add a key in Hyper Protect Crypto Services

Navigate to your instance of Hyper Protect Crypto Services and initialize the service instance. Once a master key has been created, generate or enter a root key.

Grant service authorization

Authorize Hyper Protect Crypto Services for use with IBM COS:

Open your IBM Cloud dashboard.
From the menu bar, click Manage > Access (IAM).
In the side navigation, click Authorizations.
Click Create to create an authorization.
In the Source service menu, select Cloud Object Storage.
In the Source service instance menu, select the service instance to authorize.
In the Target service menu, select Hyper Protect Crypto Services.
In the Target service instance menu, select the service instance to authorize.
Enable the Reader role.
Click Authorize.

Create a bucket

When your key exists in Hyper Protect Crypto Services and you authorized the service for use with IBM COS, you can now associate the key with a new bucket:

Navigate to your instance of Object Storage.
Click Create bucket.
Select Custom bucket.
Enter a bucket name, select the resiliency (only Regional and US Cross Region are currently supported), and choose a location and storage class.
In Service integrations, toggle Key management disabled to enable encryption key management and click on Use existing instance.
Select the associated service instance and key, and click Associate key.
Verify the information is correct.
Click Create.

You can choose to use Hyper Protect Crypto Services to manage encryption for a bucket only at the time of creation. It isn't possible to change an existing bucket to use Hyper Protect Crypto Services.

If bucket creation fails with a 400 Bad Request error with the message The Key CRN could not be found, ensure that the CRN is correct and that the service to service authorization policy exists.

In the Buckets listing, the bucket now has a View link under Attributes, indicating that the bucket has a Hyper Protect Crypto Services key enabled. To view the key details (along with other object metadata), click View.

Note that the Etag value returned for objects encrypted using Hyper Protect Crypto Services will be the actual MD5 hash of the original decrypted object.

It is also possible to use the REST API or SDKs (Go, Java, Node.js, or Python).

Creating Cross Region buckets

Creating COS Cross Region bucket with a root key from a Hyper Protect Crypto Services instance requires that instance to be configured with failover configuration.

You can confirm that failover is properly configured for the selected Hyper Protect Crypto Services instance correctly using either the IBM Cloud console or CLI.

From IBM Cloud console, navigate to a Hyper Protect Crypto Services instance and click on Overview. A "Failover" section will indicate the status of crypto units in the corresponding failover regions.

Ensure the failover section is present, all validation checks are green and there are no warnings for that Hyper Protect Crypto Services instance. If you see any errors or warnings, or if the failover section is not present, refer to the Hyper Protect Crypto Services documentation for further guidance.

You can also use the CLI to list all the crypto units for all instances belong to the targeted resource group:

ibmcloud tke cryptounits

To get status of crypto units for the selected instance, create a list of crypto units associated with that instance and compare them:

ibmcloud tke cryptounit-add

After the units have been selected, you can check their verification patterns:

ibmcloud tke cryptounit-compare

Make sure all are valid and have same verification pattern.

Once the presence of the failover configuration is verified, you may proceed to create the Cross Region bucket using the key from that Hyper Protect Crypto Services instance.

If the Cross Region bucket creation in US Cross Region with a Hyper Protect Crypto Services root key fails with a 500 error, then the user is advised to check if the status of failover configuration for that Hyper Protect Crypto Services instance (using the methods detailed above) before reattempting the bucket creation.

Key lifecycle management

Hyper Protect Crypto Services offers various ways to manage the lifecycle of encryption keys. For more details, see the Hyper Protect Crypto Services documentation.

Rotating Keys

Key rotation is an important part of mitigating the risk of a data breach. Periodically changing keys reduces the potential data loss if the key is lost or compromised. The frequency of key rotations varies by organization and depends on a number of variables, such as the environment, the amount of encrypted data, classification of the data, and compliance laws. The National Institute of Standards and Technology (NIST) provides definitions of appropriate key lengths and provides guidelines for how long keys should be used.

For more information, see the documentation for rotating keys in Hyper Protect Crypto Services.

Disabling and re-enabling keys

As an admin, you might need to temporarily disable a root key if you suspect a possible security exposure, compromise, or breach with your data. When you disable a root key, you suspend its encrypt and decrypt operations. After confirming that a security risk is no longer active, you can reestablish access to your data by enabling the disabled root key.

Deleting keys and cryptographic erasure

Cryptographic erasure (or crypto-shredding) is a method of rendering encrypted data unreadable by deleting the encryption keys rather than the data itself. When a root key is deleted in Hyper Protect Crypto Services, it will affect all objects in any buckets created using that root key, effectively "shredding" the data and preventing any further reading or writing to the buckets. This process is not instantaneous, but occurs within about 90 seconds after the key is deleted.

Although objects in a crypto-shredded bucket can not be read, and new object can not be written, existing objects will continue to consume storage until they are deleted by a user.

Restoring a deleted key

As an admin, you might need to restore a root key that you imported to Hyper Protect Crypto Services so that you can access data that the key previously protected. When you restore a key, you move the key from the Destroyed to the Active key state, and you restore access to any data that was previously encrypted with the key. This must occur within 30 days of deleting a key.

If a key that was originally uploaded by a user is deleted, and then restored using different key material, it will result in a loss of data. It is recommended to keep n-5 keys archived somewhere to ensure that the correct key material is available for restoration.

Activity Tracking

When Hyper Protect Crypto Services root keys are deleted, rotated, suspended, enabled, or restored, an Activity Tracker management event (cloud-object-storage.bucket-key-state.update) is generated in addition to any events logged by Hyper Protect Crypto Services.

In the event of a server-side failure in a lifecycle action on a key, that failure is not logged by COS. If Hyper Protect Crypto Services does not receive a success from COS for the event handling within four hours of the event being sent, Hyper Protect Crypto Services will log a failure.

The cloud-object-storage.bucket-key-state.update actions are triggered by events taking place in Hyper Protect Crypto Services, and require that the bucket is registered with the Hyper Protect Crypto Services service. This registration happens automatically when a bucket is created with a Hyper Protect Crypto Services root key.

Buckets created before February 26th, 2020 are not registered with the Hyper Protect Crypto Services service and will not receive notifications of encryption key lifecycle events at this time. These buckets can be identified by performing a bucket listing operation and looking at the dates for bucket creation. To ensure that these buckets have the latest key state from Hyper Protect Crypto Services, it is recommended that some data operation is performed, such as a PUT, GET, or HEAD on an object in each affected bucket. It is recommended that an object operation is done twice, at least an hour apart, to ensure that the key state is properly in synchronization with the Hyper Protect Crypto Services state.

For more information on Activity Tracker events for object storage, see the reference topic.