Authentication
IBM Cloud® Data Engine is deprecated. As of 18 February 2024 you can't create new instances, and access to free instances will be removed. Existing Standard plan instances are supported until 18 January 2025. Any instances that still exist on that date will be deleted.
The {{site.data.keyword.sqlquery}} service is tightly integrated with IBM Cloud® Identity and Access Management (IAM). In order to perform an action by using the Data Engine user interface or API, you require an IAM user ID, which is an IBMid with an IBM Cloud account. Data Engine then verifies whether you are authorized to work with the service instance.
Cloud Identity and Access Management (IAM) session support
The Data Engine UI supports IAM sessions. If you limit the duration of active sessions, the authentication of the UI will stop after the session expires, so you cannot submit new queries or retrieve result data until you reload the page and reauthenticate. Result data that is already cached in the browser remains visible. If you terminate an active IAM session before it expires, the Data Engine UI remains authenticated for the leftover lifetime of the current access token, which can up to 20 minutes.
Queries that are submitted from the UI can run at maximum for 1 hour or the remaining session lifetime, whichever is shorter. Queries that are submitted through the API can always run for 1 hour maximum if you retrieve the API token by using the method described in the Data Engine API documentation.
Authenticating access to data resources in Data Engine
SSO through Cloud Identity and Access Management
IAM is also the preferred and the default mechanism for authenticating access to downstream data resources that are referenced by your SQL queries. If you use IAM, ensure that the user ID that you use to submit the SQL query through the web
console or the API is authorized to read the input locations (referenced in the FROM clauses of the query), and to write the result set to the target location (referenced in the INTO clause of the query). This mechanism
provides seamless single sign-on (SSO) for your query submissions in Data Engine. You can use the following alternative methods for authenticating access to data resources:
IBM Cloud service credentials
You can use the unique CRN of a service instance in IBM Cloud to address a resource location. The credentials for accessing this data resource are retrieved from the Credentials object of that service instance. The IAM user ID that is used to submit the query must have the operator role for the service instance of the corresponding data resource.
Custom user ID and password for each data resource
You can use the USER and PASSWORD keywords in the corresponding FROM or INTO clauses to securely pass user ID and password for each resource location. To ensure that the passing of sensitive data is secure, store the password as an arbitrary secret in an instance of IBM Cloud® Secrets Manager to which you have access, and then use the CRN of this arbitrary secret instead of the plain text password. For more information, see Storing arbitrary secrets about how to store the password securely.
In addition, you also have the option to store the password as a custom standard key in an instance of IBM® Key Protect to which you have access, and then use the CRN of this custom standard key instead of the plain text password. For more information, see Setting up custom secrets in Key Protect about how to store the password securely. This option will be deprected in the future.
Custom API key for each data resource
As an alternative to providing user and password combinations, in IBM Cloud you can also securely pass API keys for each resource location. Using the APIKEY keyword inside the according FROM or INTO clauses, you can pass in this information. To ensure that the passing of sensitive data is secure, first store the API key as an IAM credential in a Secrets Manager service instance to which you have access, and then use the CRN of this
IAM credential instead of the plain text password. For more information, see Creating IAM credentials on how to store the API
key securely and the options for rotating the API key automatically.
You also have the option to store the API key as a custom standard key in a Key Protect service instance to which you have access, and then use the CRN of this custom standard key instead of the plain text password. For more information, see Setting up custom secrets in Key Protect on how to store the API key securely. This option will be deprected in the future.
Supported authentication methods per data resource for Data Engine
| Authentication method | IBM Cloud® Object Storage | IBM® Db2® on Cloud | IBM® Db2® Warehouse on Cloud |
|---|---|---|---|
| IAM SSO | Yes | Enterprise | Enterprise |
| CRN with Service Credentials | No | Yes | Yes |
| User and password through Secrets Manager CRN | No | Yes | Yes |
| User and password through Key Protect CRN | No | Yes | Yes |
| API key through Secrets Manager CRN | No | Enterprise | Enterprise |
| API key through Key Protect CRN | No | Enterprise | Enterprise |