Securing your data in Data Engine
IBM Cloud® Data Engine is deprecated. As of 18 February 2024 you can't create new instances, and access to free instances will be removed. Existing Standard plan instances are supported until 18 January 2025. Any instances that still exist on that date will be deleted.
To ensure that you can securely manage your data when you use Data Engine, it is important to know exactly what data is stored and encrypted and how you can delete any stored data. Depending on your security requirements, you can encrypt data with customer-managed keys by integrating with IBM Cloud key management services such as Key Protect, which supports the Bring Your Own Key (BYOK) method.
How your data is stored and encrypted in Data Engine
All data that is stored in Data Engine has by default service-managed encryption. You can also choose to encrypt with BYOK using IBM® Key Protect for IBM Cloud®. SQL query text and error messages can be encrypted by associating a IBM® Key Protect for IBM Cloud® during instance creation. Table metadata is encrypted by using the same mechanism for all tables or views that were created after August 23rd, 2022. All table metadata that was created before that date is encrypted by using service-managed encryption. You can re-create tables to convert the encryption method to BYOK.
The following elements of table metadata cannot be encrypted with BYOK by design: table names and partition names that are generated from values of a partitioning column.
Do not include PII or sensitive data in table names, and do not use columns containing PII or sensitive data as partitioning columns.
If you are processing sensitive data, ensure that the actual data storage locations on which Data Engine works (for example COS buckets, Event Streams queues, or Db2 tables) are adequately protected. Refer to the documentation of the respective cloud service for configuring security for these storage locations.
Protecting your sensitive data in Data Engine
You can add a higher level of encryption protection and control to your data at rest (when it is stored) by enabling integration with Key Protect. The data that you store in IBM Cloud is encrypted at rest by using a service-managed key. If you need to control the encryption keys, you can integrate Key Protect. This process is commonly referred to as Bring your own keys (BYOK). With Key Protect you can create, import, and manage encryption keys. You can assign access policies to the keys, assign users or service IDs to the keys, or give the key access only to a specific service. The first 20 keys are free.
Enabling customer-managed keys for Data Engine
Follow the steps to set up Key Protect encryption when you create your instance.
Deleting your data in Data Engine
Your data is deleted 30 days after you delete an instance. If you use Key Protect integration, you can destroy your data immediately by destroying the key in Key Protect.
Deleting Data Engine instances
Deleting a key or the containing Key Protect instance from the system will shred its contents and any data still encrypted with that key. When it is removed, it cannot be undone or reversed. A Data Engine instance that is associated with a deleted key cannot be used for any further queries. You need to create a new Data Engine instance for future queries.
The Data Engine data retention policy describes how long your data is stored after you delete the service. The data retention policy is included in the Data Engine service description, which you can find in the IBM Cloud Terms and Notices.