IBM Cloud Docs
Encrypting SQL queries with Key Protect

Encrypting SQL queries with Key Protect

IBM Cloud® Data Engine is deprecated. As of 18 February 2024 you can't create new instances, and access to free instances will be removed. Existing Standard plan instances are supported until 18 January 2025. Any instances that still exist on that date will be deleted.

By default, Data Engine uses server-managed encryption at rest for all job information that is recorded about your stored queries. If you are processing sensitive data in your queries that is governed by special regulations, you can additionally use customer-managed keys to encrypt the SQL query texts and error messages that are stored in the job information.

IBM® Key Protect for IBM Cloud® is a centralized key management system (KMS) for generating, managing, and destroying encryption keys that are used by IBM Cloud® services. You can associate a key that is managed in IBM Key Protect with an SQL query instance to encrypt your queries. Customer key encryption can be configured only when you are creating the SQL query instance, and that configuration cannot be changed later. However, you can always create a new SQL query instance with a different configuration and use that for future queries. Encryption is only available for instances based on the Data Engine Standard plan.

About customer-managed keys

Data Engine uses envelope encryptionThe process of encrypting data with a data encryption key and then encrypting the key with a root key that can be fully managed. to implement customer-managed keys. Envelope encryption describes encrypting one encryption key with another encryption key. The key used to encrypt the actual data is known as a data encryption key (DEK)A cryptographic key used to encrypt data that is stored in an application.. The DEK itself is never stored but is wrapped by a second key that is known as the key encryption key (KEK) to create a wrapped DEK. To decrypt data, the wrapped DEK is unwrapped to get the DEK. This process is possible only by accessing the KEK, which in this case is your root key that is stored in Key Protect.

You own the KEK, which you create as a root key in the Key Protect service. The Data Engine service never sees the root (KEK) key. Its storage, management, and use to wrap and unwrap the DEK is performed entirely within the key management service. If you disable or delete the key, the data can no longer be decrypted.

Key Protect keys are secured by FIPS 140-2 Level 3 certified cloud-based hardware security modules (HSMs)A physical appliance that provides on-demand encryption, key management, and key storage as a managed service.. For more information, see Bringing your encryption keys to the cloud.

Working with customer-managed keys

You can use IBM Cloud® Activity Tracker to audit the lifecycle events of your keys, such as creating a key, deleting a key, rotating a key, and more. For more information, see IBM Cloud Activity Tracker events for Key Protect.

Setting up Key Protect encryption

  1. Provision Key Protect on your IBM Cloud account.

  2. Go to your instance of Key Protect and generate or enter a root key.

  3. Create a new Standard plan instance of Data Engine and select the Key Protect key for encrypting your queries. For performance reasons, create both the Key Protect and Data Engine instances in the same IBM Cloud region.

  4. Give the new Data Engine instance access to your Key Protect key.

    • Open your IBM Cloud dashboard.
    • From the menu bar, click Manage > Access (IAM), and select Authorizations.
    • Click Create.
    • In the source service menu, select IBM Cloud Data Engine.
    • In the source service instance menu, select the service instance to authorize.
    • In the target service menu, select IBM Key Protect.
    • In the target service instance menu, select the service instance to authorize.
    • Enable the Reader role.
    • Click Authorize.

Further considerations

Customer key encryption in Data Engine applies to the queries that you are processing. With each query, you explicitly specify IBM Cloud® Object Storage locations for input and target data, which is not controlled by Data Engine. So, if you are processing sensitive data, make sure that your query results are written to an IBM Cloud Object Storage location that has appropriate protection. Default target locations, where results are stored by default if no other result locations are specified, are not and cannot be encrypted.

IBM Cloud Object Storage documents how to configure customer key encryption for the Cloud Object Storage buckets in Managing encryption storing the actual data.

If you use Key Protect to encrypt your queries, IBM staff cannot view the encrypted query texts and error messages. Therefore, provide this data explicitly to IBM service in a support case, so you can make sure that no sensitive information is exposed.

Deleting a key or an instance

Deleting a key or the containing Key Protect instance from the system will shred its contents and any data still encrypted with that key. When it is removed, it cannot be undone or reversed. A Data Engine instance that is associated with a deleted key cannot be used for any further queries. You need to create a Data Engine instance for future queries.