Managing encryption in Event Streams
By default, message payload data in IBM® Event Streams for IBM Cloud® is encrypted at rest by using a randomly generated key. Although this default encryption model provides at-rest security, you might need a higher level of control. For these use cases, Event Streams supports customer-managed encryption with the following IBM Cloud® key management services:
- IBM® Key Protect for IBM Cloud® (Bring Your Own Key - BYOK) helps you provision encrypted keys for apps across IBM Cloud services. As you manage the lifecycle of your keys, you benefit from knowing that your keys are secured by FIPS 140-2 Level three certified cloud-based hardware security modules (HSMs) that protect against the theft of information. For more information about using Key Protect, see the Getting Started tutorial.
- Hyper Protect Crypto Services (Keep Your Own Key - KYOK) is a single-tenant, dedicated HSM that is controlled by you. The service is built on FIPS 140-2 Level 4-certified hardware, the highest offered by any cloud provider in the industry. For more information about using Hyper Protect Crypto Services, see the Getting Started tutorial.
These services allow the use of a customer-provided key to control encryption. By disabling or deleting this key, you can prevent any further access to the data that is stored by the service, because it is no longer possible to decrypt it.
Consider to use customer-managed keys, if you require the following features:
- Encryption of data at rest controlled by your own key.
- Explicit control of the lifecycle of data that is stored at rest.
Customer-managed keys are available on the Enterprise plan and only on clusters that were created after October 2019.
Deletion of the customer-managed key is unrecoverable and results in the loss of any data that is stored in your Event Streams instance.
What is not covered by customer-managed encryption
If customer-managed encryption feature is selected, be aware that only message payload data is covered by this encryption. Event Streams encrypts at rest other data that is related to the use of the service. However, although encrypted, non-message payload data is not encrypted with the customer-managed encryption. Examples are client metadata such as topic names, topic configuration data, schemas stored in the schema registry and metadata that is stored in relation to the configuration of the Enterprise instance.
Therefore, do not use confidential information in such client metadata.
How customer-managed encryption works
Event Streams uses a concept that is called envelope encryption to implement customer-managed keys.
Envelope encryption is the practice of encrypting one encryption key with another encryption key. The key used to encrypt the actual data is known as a data encryption key (DEK). The DEK itself is never stored, but instead is wrapped by a second key that is known as the key encryption key (KEK) to create a wrapped DEK.
To decrypt data, the wrapped DEK must first be unwrapped to get the DEK. This process is possible only by accessing the KEK, which in this case is your root key that is stored in either Key Protect or Hyper Protect Crypto Services.
You own the KEK, which you create as a root key in the Hyper Protect Crypto Services or Key Protect service. The Event Streams service never sees the root (KEK) key. Its storage, management, and use to wrap and unwrap the DEK is performed entirely within the key management service. If you disable or delete the key, the data can no longer be decrypted.
Enabling a customer-managed key for Event Streams
Complete the following steps to provision your Event Streams instance to use a customer-managed key:
- Provision an instance of Key Protect or Hyper Protect Crypto Services.
- Create an authorization policy to allow the Event Streams service to access the key management service instance as a Reader. For more information, see Using authorizations to grant access between services.
- Create or import a root key into your key management service instance.
- Retrieve the Cloud Resource Name (CRN) of the key by using the View CRN option in the key management service instance GUI.
- Provision an instance of Event Streams. This feature is supported on the Enterprise plan only.
If provisioning through IBM Cloud® console, select a key management service instance and then select a root key from the instance.
If provisioning through CLI, use the following command:
ibmcloud resource service-instance-create EVENT-STREAMS-INSTANCE-NAME messagehub ibm.message.hub.enterprise.3nodes.2tb REGION -p '{"kms_key_crn":"KMS_KEY_CRN"}'
If you want to update your existing Event Streams instance to use a customer-managed key, open a support ticket on Event Streams that contains the following information:
- The CRN of the root key that you want to use in your key management service instance.
- The CRN of your Event Streams service instance.
You find this CRN by copying and pasting the full IBM Cloud console URL after you click the Event Streams service in the console. Alternatively, paste in the output from the following CLI command:
ibmcloud resource service-instance NAME
The Event Streams Operations team responds to your support ticket to confirm that your instance of IBM Cloud is now using a customer-managed key. Expect the enablement to be completed in one business day.
This operation is destructive and results in the loss of all message and topic definitions. For more information, see deciding to enable customer-managed keys.
Using a customer-managed key
After a customer-managed key is enabled, the cluster operates as normal, but with the following additional capabilities.
Preventing access to data
To temporarily prevent access, disable your root key. As a consequence, Event Streams can no longer access the data because it can no longer access the key.
To remove access permanently, delete the key. However, you must take extreme caution because this operation is non-recoverable. You lose access to any data that is stored in your Event Streams instance. It is not possible to recover this data.
In both cases, the Event Streams instance shuts down and no longer accepts or processes connections. An Activity Tracker event is generated to report the action. For more information, see Activity Tracker events.
The authorization is to be left in place between your Event Streams and the key management service instance at all times even after deleting the service instance. Event Streams will continue to need the service to service policy in place to deregister the key. To determine when it is safe to remove the service-to-service policy, check the associated resources on the KMS root key. If the root key is still associated with the Event Streams instance, leave the policy in place. While removing this authorization prevents Event Streams from future access to your data, already in-use data continues to be available for a period of time.
You are charged for your instance of Event Streams until you deprovision it using the IBM Cloud console or CLI. These charges are still applied even if you chose to prevent access to your data.
Restoring access to data
Access can be restored only if the key was not deleted. To restore access, re-enable your root key. After a short period of initialization, your Event Streams instance is restarted and starts accepting connections again. All data is retained, subject to the normal retention limits configured in your instance.
An activity tracker event is generated to report the action. For more information, see Activity Tracker events.
Rotating the key
Key Protect and Hyper Protect Crypto Services support the rotation of root keys, either on demand or on a schedule. When rotating the key, Event Streams adopts the new key by rewrapping the DEK as described previously in how customer-managed encryption works.
An activity tracker event is generated to report the action. For more information, see Activity Tracker events.
Disabling customer-managed encryption
After you enable customer-managed encryption, it is not possible to disable it. Instead, you must delete the service instance and create a new instance.