watsonx.data | IBM Cloud API Docs

Introduction

Last updated: 2023-07-10

IBM® watsonx.data is a data management solution for collecting, storing, querying, and analyzing all your enterprise data (structured, semi-structured, and unstructured) with a single unified data platform. It provides a flexible and reliable platform that is optimized to work on open data formats.

The api are in beta state and expected to change.

Endpoint URLs

The Watsonx.data API endpoint URL is based on your IBM Cloud Pak deployment URL. When you call the API, add the path for each method to form the complete API endpoint for your request. For example, if your instance is deployed at https://www.example.com:31843, you can access the APIs at https://www.example.com:31843/lakehouse/api/v1/{instance_id}/{method}.

Authentication

Required Purpose: To work with the API, you must use an IBM Cloud Pak for Data API key (platform API key or instance API key), or IBM Cloud Pak for Data authorization token (platform token or instance token). The API key or token is used to determine the actions that a user or service ID has access to when they use the API.

You can generate an API key or authorization token for an authenticated user by using the Cloud Pak for Data APIs. For more information, see Generating an authorization token or API key.

To use the API, add a valid API key or token to the HTTP Authorization request header. For token, -H 'Authorization: Bearer {TOKEN}'. For API key, you need to firstly encode username together with API key: echo "<username>:<api_key>" | base64. Then, you can use the ZenApiKey header to authenticate to an endpoint: -H "Authorization: ZenApiKey <encoded value>".

To retrieve your access token:

For most Cloud Pak for Data endpoints, you can use the generated API key in the authorization header of subsequent API calls.

curl -H "Authorization: ZenApiKey <token>" "https://<cpd_instance_route>/<endpoint>"

If the ZenApiKey token is not supported, use a generated Bearer token in the authorization header of subsequent API calls.

curl -H "Authorization: Bearer <token>" "https://<cpd_instance_route>/<endpoint>"

Auditing

Required if applicable Purpose: Describes that the API generates auditing events that can be consumed by the Activity Tracker service, and links to your related product docs for more information. Required only if any of the API's methods is enabled to generate auditing events.

Example:

You can monitor API activity within your account by using the IBM Cloud Activity Tracker service. Whenever an API method is called, an event is generated that you can then track and audit from within Activity Tracker. The specific event type is listed for each individual method.

For more information about how to track Certificate Manager activity, see Auditing events for Certificate Manager.

Error handling

This API uses standard HTTP response codes to indicate whether a method completed successfully. A 200 response indicates success. A 400 type response indicates a failure, and a 500 type response indicates an internal system error.

HTTP Error Code	Description	Recovery
`200`	Success	The request was successful.
`201`	Created	The requested resource successfully created in a synchronous manner.
`204`	No Content	The server successfully processed the request and is not returning any content.
`400`	Bad Request	The input parameters in the request body are either incomplete or in the wrong format. Be sure to include all required parameters in your request.
`401`	Unauthorized	You are not authorized to make this request. Log in to IBM Cloud and try again. If this error persists, contact the account owner to check your permissions.
`403`	Forbidden	The supplied authentication is not authorized to access '{namespace}'.
`404`	Not Found	The requested resource could not be found.
`409`	Conflict	The entity is already in the requested state.
`500`	Internal Server Error	Your request could not be processed. Wait a few minutes and try again.

Pagination

Some API requests might return many results. To avoid performance issues, these results are returned one page at a time, with a limited number of results on each page. GET requests for the following resources use pagination:

/v1/statement

For the request that uses pagination, the response does include following urls to make subsequent requests :

infoUri: The URL for requesting the first page of results.
nextUri: The URL for requesting the next page of results.

Get list of registered buckets

GET /bucket_registrations

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.bucket_registration.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

curl example

curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/bucket_registrations"

Response

Response Body

BucketRegistrationCollection

List bucket registrations

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "bucket_registrations": [
    {
      "actions": [
        "browse",
        "view",
        "modify",
        "create",
        "grant",
        "revoke",
        "update",
        "remove",
        "activate",
        "register"
      ],
      "associated_catalog": {
        "catalog_name": "iceberg_catalog",
        "catalog_tags": []
      },
      "bucket_details": {
        "bucket_name": "iceberg-bucket",
        "endpoint": "http://xyz-minio-svc:9000"
      },
      "bucket_display_name": "hive-bucket",
      "bucket_id": "iceberg-bucket",
      "bucket_type": "minio",
      "created_by": "user",
      "created_on": "1699457595",
      "description": "default bucket",
      "managed_by": "ibm",
      "region": "us-south",
      "state": "active",
      "tags": [
        "tag1",
        "tag2"
      ]
    },
    {
      "actions": [
        "browse",
        "view",
        "modify",
        "create",
        "grant",
        "revoke",
        "update",
        "remove",
        "activate",
        "register"
      ],
      "associated_catalog": {
        "catalog_name": "hive_data",
        "catalog_tags": []
      },
      "bucket_details": {
        "bucket_name": "hive-bucket",
        "endpoint": "http://xyz-minio-svc:9000"
      },
      "bucket_display_name": "hive-bucket",
      "bucket_id": "hive-bucket",
      "bucket_type": "minio",
      "created_by": "user",
      "created_on": "1699457595",
      "description": "default bucket",
      "managed_by": "ibm",
      "region": "us-south",
      "state": "active",
      "tags": [
        "tag1",
        "tag2"
      ]
    }
  ]
}

Register a new bucket

POST /bucket_registrations

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.bucket_registration.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*

BucketRegistrationPrototype

Request body

curl example

curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json" -d '{ "bucket_details": { "access_key": "<access_key>", "bucket_name": "sample-bucket", "endpoint": "https://s3.{region}.cloud-object-storage.appdomain.cloud/", "secret_key": "secret_key" }, "bucket_display_name": "sample-bucket-displayname", "bucket_tags": [ "read customer data", "write customer data'" ], "bucket_type": "ibm_cos", "catalog_name": "sampleCatalog", "catalog_tags": [ "catalog_tag_1", "catalog_tag_2" ], "description": "COS bucket for customer data", "managed_by": "ibm", "region": "us-south", "state": "active", "table_type": "iceberg" }' "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/bucket_registrations"

Response

Response Body

BucketRegistration

Bucket

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "actions": [
    "create",
    "update"
  ],
  "associated_catalog": {
    "catalog_name": "hive_data",
    "catalog_type": "hive"
  },
  "bucket_display_name": "samplebucketdisplayname",
  "bucket_id": "samplebucketid",
  "bucket_name": "samplebucket",
  "bucket_type": "minio",
  "created_by": "username@domain.com",
  "created_on": "1699457595",
  "description": "default bucket",
  "endpoint": "https://s3.<region>.cloud-object-storage.appdomain.cloud/",
  "managed_by": "ibm",
  "region": "us-south",
  "state": "active",
  "tags": [
    "tag1",
    "tag2"
  ]
}

Get a registered bucket

GET /bucket_registrations/{bucket_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.bucket_registration.get

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

bucket_id
Required*
string
bucket id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/bucket_registrations/{bucket_id}"

Response

Response Body

BucketRegistration

Bucket

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "actions": [
    "create",
    "update"
  ],
  "associated_catalog": {
    "catalog_name": "hive_data",
    "catalog_type": "hive"
  },
  "bucket_display_name": "samplebucketdisplayname",
  "bucket_id": "samplebucketid",
  "bucket_name": "samplebucket",
  "bucket_type": "minio",
  "created_by": "username@domain.com",
  "created_on": "1699457595",
  "description": "default bucket",
  "endpoint": "https://s3.<region>.cloud-object-storage.appdomain.cloud/",
  "managed_by": "ibm",
  "region": "us-south",
  "state": "active",
  "tags": [
    "tag1",
    "tag2"
  ]
}

Unregister a bucket

DELETE /bucket_registrations/{bucket_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.bucket_registration.delete

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

bucket_id
Required*
string
bucket id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X DELETE -H "accept:  */*" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/bucket_registrations/{bucket_id}"

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Update bucket details & credentials

PATCH /bucket_registrations/{bucket_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.bucket_registration.update

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

bucket_id
Required*
string
bucket id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

application/json-patch+jsonBucketRegistrationPatch

Request body

Examples:

View

curl example

curl -X PATCH -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json" -d '{ "access_key": "<access_key>", "bucket_display_name": "sample-bucket-displayname", "description": "COS bucket for customer data", "secret_key": "<secret_key>", "tags": [ "testbucket", "userbucket" ] }' "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/bucket_registrations/{bucket_id}"

Response

Response Body

BucketRegistration

Bucket

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "actions": [
    "create",
    "update"
  ],
  "associated_catalog": {
    "catalog_name": "hive_data",
    "catalog_type": "hive"
  },
  "bucket_display_name": "samplebucketdisplayname",
  "bucket_id": "samplebucketid",
  "bucket_name": "samplebucket",
  "bucket_type": "minio",
  "created_by": "username@domain.com",
  "created_on": "1699457595",
  "description": "default bucket",
  "endpoint": "https://s3.<region>.cloud-object-storage.appdomain.cloud/",
  "managed_by": "ibm",
  "region": "us-south",
  "state": "active",
  "tags": [
    "tag1",
    "tag2"
  ]
}

Activate a registered bucket

POST /bucket_registrations/{bucket_id}/activate

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.bucket_registration_activate.add

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

bucket_id
Required*
string
bucket id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -d "" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/bucket_registrations/{bucket_id}/activate"

Response

Response Body

CreateActivateBucketCreatedBody

Activate bucket

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "response": {
    "message": "Activate bucket",
    "message_code": "success"
  }
}

Deactivate a bucket

DELETE /bucket_registrations/{bucket_id}/deactivate

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.bucket_registration_activate.remove

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

bucket_id
Required*
string
bucket id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X DELETE -H "accept: */*" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/bucket_registrations/{bucket_id}/deactivate"

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Fetch all objects from a given bucket

GET /bucket_registrations/{bucket_id}/objects

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.bucket_registration_object.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

bucket_id
Required*
string
bucket id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/bucket_registrations/{bucket_id}/objects"

Response

Response Body

BucketRegistrationObjectCollection

List bucket objects

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "objects": [
    "/path/file1.json",
    "/path/file2.txt",
    "/path/file3.csv",
    "/path/file4.parquet"
  ]
}

Check whether provided bucket credentials are valid or not

POST /test_bucket_connection

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.test_bucket_connection.evaluate

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*

CheckBucketCredentialsBody

Check if given credentials for the bucket are valid

curl example

curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json" -d '{ "access_key": "<access_key>", "bucket_name": "sample-bucket", "bucket_type": "ibm_cos", "endpoint": "https://s3.<region>.cloud-object-storage.appdomain.cloud/", "region": "us-south", "secret_key": "<secret_key>" }' "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/test_bucket_connection"

Response

Response Body

TestBucketConnectionOKBody

ValidateBucketRegistrationCredentials OK

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal Server Error

Example responses

Status 200

{
  "bucket_status": {
    "state": true,
    "state_message": "Credentials provided for <bucket-name> bucket are correct."
  },
  "response": {
    "message": "Test bucket connection",
    "message_code": "success"
  }
}

Add or create a new database with driver

POST /database_driver_registrations

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.database_driver_registration.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Form Parameters

curl example

curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: multipart/form-data" -F "port = t" -F "driver = " -F "database_display_name = t" -F "ssl = " -F "database_name =" -F "catalog_name =t" -F "hostname =t" -F "username =" -F "created_on =" -F "driver_file_name =" -F "certificate =" -F "certificate_extension =" -F "password =" -F "description =" -F "database_type =t" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/database_driver_registrations"

Response

Response Body

DatabaseRegistration

database registration object

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "actions": [
    "update",
    "delete"
  ],
  "associated_catalog": {
    "catalog_name": "iceberg_data",
    "catalog_type": "iceberg"
  },
  "created_by": "user1@bim.com",
  "created_on": "1686792721",
  "database_details": {
    "database_name": "new_database",
    "hostname": "netezza://abc.efg.com",
    "password": "samplepassword",
    "port": 4353,
    "ssl": true,
    "tables": "netezza_table_name",
    "username": "sampleuser"
  },
  "database_display_name": "new_database",
  "database_id": "new_database_id",
  "database_type": "netezza",
  "description": "Description of the database",
  "tags": [
    "testdatabase",
    "userdatabase"
  ]
}

Get list of databases

GET /database_registrations

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.database_registration.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

curl example

curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/database_registrations"

Response

Response Body

DatabaseRegistrationCollection

list database registrations

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "database_registrations": [
    {
      "actions": [
        "update",
        "delete"
      ],
      "associated_catalog": {
        "catalog_name": "hive_data",
        "catalog_tags": [
          "catalog_tag_1",
          "catalog_tag_2"
        ],
        "catalog_type": "hive"
      },
      "created_by": "user1@bim.com",
      "created_on": "1686792721",
      "database_details": {
        "database_name": "new_database",
        "hostname": "netezza://ps.fyre.com",
        "password": "samplepassword",
        "port": 4353,
        "ssl": true,
        "tables": "netezza_table_name",
        "username": "sampleuser"
      },
      "database_display_name": "new_database",
      "database_id": "new_database_id",
      "database_type": "netezza",
      "description": "Description of the external Database",
      "tags": [
        "testdatabase",
        "userdatabase"
      ]
    }
  ]
}

Add or create a new database

POST /database_registrations

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.database_registration.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*

DatabaseRegistrationPrototype

Request body

curl example

curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json" -d '{ "catalog_name": "sampleCatalog", "created_on": 0, "database_details": { "certificate": "contents of a pem/crt file", "certificate_extension": "pem/crt", "database_name": "new_database", "hostname": "db2@<hostname>.com", "hosts": "abc.com:1234,xyz.com:4321", "password": "samplepassword", "port": 4553, "sasl": true, "ssl": true, "tables": "kafka_table_name", "username": "sampleuser" }, "database_display_name": "new_database", "database_properties": [ { "encrypt": true, "key": "abc", "value": "xyz" } ], "database_type": "db2", "description": "db2 extenal database description", "tags": [ "tag_1", "tag_2" ] }' "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/database_registrations"

Response

Response Body

DatabaseRegistration

database registration object

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "actions": [
    "update",
    "delete"
  ],
  "associated_catalog": {
    "catalog_name": "iceberg_data",
    "catalog_type": "iceberg"
  },
  "created_by": "user1@bim.com",
  "created_on": "1686792721",
  "database_details": {
    "database_name": "new_database",
    "hostname": "netezza://abc.efg.com",
    "password": "samplepassword",
    "port": 4353,
    "ssl": true,
    "tables": "netezza_table_name",
    "username": "sampleuser"
  },
  "database_display_name": "new_database",
  "database_id": "new_database_id",
  "database_type": "netezza",
  "description": "Description of the database",
  "tags": [
    "testdatabase",
    "userdatabase"
  ]
}

Get a registered databases

GET /database_registrations/{database_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.database_registration.get

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

database_id
Required*
string
database id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/database_registrations/{database_id}"

Response

Response Body

DatabaseRegistration

database registration object

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "actions": [
    "update",
    "delete"
  ],
  "associated_catalog": {
    "catalog_name": "iceberg_data",
    "catalog_type": "iceberg"
  },
  "created_by": "user1@bim.com",
  "created_on": "1686792721",
  "database_details": {
    "database_name": "new_database",
    "hostname": "netezza://abc.efg.com",
    "password": "samplepassword",
    "port": 4353,
    "ssl": true,
    "tables": "netezza_table_name",
    "username": "sampleuser"
  },
  "database_display_name": "new_database",
  "database_id": "new_database_id",
  "database_type": "netezza",
  "description": "Description of the database",
  "tags": [
    "testdatabase",
    "userdatabase"
  ]
}

Delete a database

DELETE /database_registrations/{database_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.database_registration.delete

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

database_id
Required*
string
database id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X DELETE -H "accept: */*" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/database_registrations/{database_id}"

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Update database details

PATCH /database_registrations/{database_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.database_registration.update

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

database_id
Required*
string
database id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

application/json-patch+jsonDatabaseRegistrationPatch

Request body

curl example

curl -X PATCH -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json-patch+json" -d '{ "database_details": { "password": "samplepassword", "username": "sampleuser" }, "database_display_name": "new_database", "description": "External database description", "tags": [ "testdatabase", "userdatabase" ] }' "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/database_registrations/{database_id}"

Response

Response Body

DatabaseRegistration

database registration object

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "actions": [
    "update",
    "delete"
  ],
  "associated_catalog": {
    "catalog_name": "iceberg_data",
    "catalog_type": "iceberg"
  },
  "created_by": "user1@bim.com",
  "created_on": "1686792721",
  "database_details": {
    "database_name": "new_database",
    "hostname": "netezza://abc.efg.com",
    "password": "samplepassword",
    "port": 4353,
    "ssl": true,
    "tables": "netezza_table_name",
    "username": "sampleuser"
  },
  "database_display_name": "new_database",
  "database_id": "new_database_id",
  "database_type": "netezza",
  "description": "Description of the database",
  "tags": [
    "testdatabase",
    "userdatabase"
  ]
}

API to validate the database connection

POST /test_database_connection

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.test_database_connection.evaluate

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*

ValidateDatabaseBody

Request body

curl example

curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json" -d '{ "certificate": "contents of a pem/crt file", "database_details": { "database_name": "sampledatabase", "hostname": "db2@hostname.com", "password": "samplepassword", "port": 4553, "sasl": true, "ssl": true, "tables": "kafka_table_name", "username": "sampleuser" }, "database_type": "netezza", }' "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/test_database_connection"

Response

Response Body

TestDatabaseConnectionResponse

Success response

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "connection_response": {
    "state": true,
    "state_message": "connection successful"
  }
}

Get all engine details

GET /engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.engines.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

curl example

curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/engines"

Response

Response Body

Engines

List all engines

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "engines": {
    "db2_engines": [],
    "milvus_engines": [],
    "netezza_engines": [],
    "prestissimo_engines": [],
    "presto_engines": [],
    "spark_engines": []
  }
}

Get list of all deployments

GET /instance

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.instance.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

curl example

curl -X 'GET' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/instance' -H 'accept: application/json' -H "AuthInstanceId: {instance id}"

Response

Response Body

GetDeploymentsOKBody

Response body structure for get deployments

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "deploymentresponse": {
    "deployment": {
      "cloud_type": "aws",
      "enable_private_endpoints": true,
      "enable_public_endpoints": true,
      "first_time_use": false,
      "formation_id": "new_form_id",
      "id": "dep_id",
      "plan_id": "new_plan_id",
      "platform_options": {
        "backup_encryption_key_crn": "2nf8f8b3kd8wknfkf",
        "disk_encryption_key_crn": "hjdkd8wjnnd93ujd9",
        "key_protect_key_id": "8ndkenkwjdciendj"
      },
      "region": "us-south",
      "type": "deployment_type",
      "version": "1.0.2"
    }
  },
  "response": {
    "message": "get instance",
    "message_code": "success"
  }
}

list all other engine details

GET /other_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.other_engine.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

curl example

curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/other_engines"

Response

Response Body

OtherEngineCollection

list other engines

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "other_engines": [
    {
      "created_by": "<username>@<domain>.com",
      "created_on": 163788384993,
      "description": "other engine for running queries",
      "engine_details": {
        "connection_string": "https://other-connection-string-sample.com",
        "engine_type": "netezza"
      },
      "engine_display_name": "sampleEngine",
      "engine_id": "sampleEngine123",
      "origin": "external",
      "status": "registered",
      "type": "other"
    }
  ]
}

Create a new engine

POST /other_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.other_engine.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*

OtherEnginePrototype

Create other Engine Body

curl example

curl -X POST -H "content-type: application/json" -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -d '{ "description": "external engine description", "engine_details": { "connection_string": "1.2.3.4", "engine_type": "netezza", "metastore_host": "1.2.3.4" }, "engine_display_name": "sampleEngine01", "tags": [ "tag1", "tag2" ] }' "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/other_engines"

Response

Response Body

OtherEngine

external engine details

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "OtherEngine": {
    "created_by": "<username>@<domain>.com",
    "created_on": 163788384993,
    "description": "other engine for running sql queries",
    "engine_display_name": "sampleEngine",
    "engine_id": "sampleEngine123",
    "origin": "external",
    "status": "registered",
    "type": "other"
  }
}

Delete an engine from lakehouse

DELETE /other_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.other_engine.delete

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X DELETE -H "accept: */*" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/other_engines/{engine_id}"

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Get list of all db2 engines

GET /db2_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.db2_engine.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

curl example

curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/db2_engines"

Response

Response Body

Db2EngineCollection

list db2 engines

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "db2_engines": [
    {
      "created_by": "user@test.com",
      "created_on": 1700322436,
      "description": "db2 engine for running sql queries",
      "engine_details": {
        "connection_string": "jdbc:db2://<hostname>:<port>/<database>",
        "metastore_host": "thrift://mh-connection-string-sample.com"
      },
      "engine_display_name": "db2",
      "engine_id": "db2505",
      "origin": "external",
      "status": "REGISTERED",
      "tags": [
        "tag1",
        "tag2"
      ],
      "type": "db2"
    }
  ]
}

Create a new db2 engine

POST /db2_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.db2_engine.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*

Db2EnginePrototype

Create db2 engine body

curl example

curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json" -d '{ "description": "db2 engine description", "engine_details": { "connection_string": "1.2.3.4" }, "engine_display_name": "sampleEngine", "origin": "external", "tags": [ "tag1", "tag2" ], "type": "db2" }' "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/db2_engines"

Response

Response Body

Db2Engine

Db2 engine details

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "Db2Engine": null,
  "created_by": "user@test.com",
  "created_on": 1700322469,
  "description": "updated description for db2 engine.",
  "engine_details": {
    "connection_string\"": "jdbc:db2://localhost:5480/database",
    "metastore_host": "thrift://mh-connection-string-sample.com"
  },
  "engine_display_name": "sample db2 Engine Display Name",
  "engine_id": "sample db2 Engine Name",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "db2"
}

Delete a db2 engine

DELETE /db2_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.db2_engine.delete

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X DELETE -H "accept: */*" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/db2_engines/{engine_id}"

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Update details of db2 engine

PATCH /db2_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.db2_engine.update

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

application/json-patch+jsonDb2EnginePatch

Update Engine Body

curl example

curl -X PATCH -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json-patch+json" -d '{ "description": "db2 engine updated description", "engine_display_name": "sampleEngine", "tags": [ "tag1", "tag2" ] }' "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/db2_engines/{engine_id}"

Response

Response Body

Db2Engine

Db2 engine details

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "Db2Engine": null,
  "created_by": "user@test.com",
  "created_on": 1700322469,
  "description": "updated description for db2 engine.",
  "engine_details": {
    "connection_string\"": "jdbc:db2://localhost:5480/database",
    "metastore_host": "thrift://mh-connection-string-sample.com"
  },
  "engine_display_name": "sample db2 Engine Display Name",
  "engine_id": "sample db2 Engine Name",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "db2"
}

Get list of all netezza engines

GET /netezza_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.netezza_engine.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

curl example

curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/netezza_engines"

Response

Response Body

NetezzaEngineCollection

list Netezza engines

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "netezza_engines": [
    {
      "created_by": "user@test.com",
      "created_on": 1700322469,
      "description": "netezza engine for running sql queries",
      "engine_details": {
        "connection_string": "jdbc:netezza://localhost:5480/database",
        "metastore_host": "thrift://mh-connection-string-sample.com"
      },
      "engine_display_name": "netezza",
      "engine_id": "netezza170",
      "origin": "external",
      "status": "REGISTERED",
      "tags": [
        "tag1",
        "tag2"
      ],
      "type": "netezza"
    }
  ]
}

Create a new netezza engine

POST /netezza_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.netezza_engine.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*

NetezzaEnginePrototype

Create netezza engine body

curl example

curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json" -d '{ "description": "netezza engine description", "engine_details": { "connection_string": "1.2.3.4" }, "engine_display_name": "sampleEngine", "origin": "external", "tags": [ "tag1", "tag2" ], "type": "netezza" }' "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/netezza_engines"

Response

Response Body

NetezzaEngine

Netezza engine details

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "NetezzaEngine": null,
  "created_by": "user@test.com",
  "created_on": 1700322469,
  "description": "updated description for netezza engine.",
  "engine_details": {
    "connection_string\"": "jdbc:netezza://localhost:5480/database",
    "metastore_host": "thrift://mh-connection-string-sample.com"
  },
  "engine_display_name": "sample Netezza Engine Display Name",
  "engine_id": "sample Netezza Engine Name",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "netezza"
}

Delete a netezza engine

DELETE /netezza_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.netezza_engine.delete

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X DELETE -H "accept: */* " -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/netezza_engines/{engine_id}"

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Update details of netezza engine

PATCH /netezza_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.netezza_engine.update

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

application/json-patch+jsonNetezzaEnginePatch

Update Engine Body

curl example

curl -X PATCH -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json-patch+json" -d '{ "description": "netezza engine updated description", "engine_display_name": "sampleEngine", "tags": [ "tag1", "tag2" ] }' "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/netezza_engines/{engine_id}"

Response

Response Body

NetezzaEngine

Netezza engine details

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "NetezzaEngine": null,
  "created_by": "user@test.com",
  "created_on": 1700322469,
  "description": "updated description for netezza engine.",
  "engine_details": {
    "connection_string\"": "jdbc:netezza://localhost:5480/database",
    "metastore_host": "thrift://mh-connection-string-sample.com"
  },
  "engine_display_name": "sample Netezza Engine Display Name",
  "engine_id": "sample Netezza Engine Name",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "netezza"
}

Get list of all prestissimo engines

GET /prestissimo_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.prestissimo_engine.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Response

Response Body

PrestissimoEngineCollection

list Prestissimo Engines

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "prestissimo_engines": [
    {
      "actions": [
        "view",
        "use",
        "update",
        "select",
        "access_ui",
        "associate",
        "disassociate",
        "restart",
        "pause",
        "resume",
        "grant",
        "revoke",
        "delete",
        "create",
        "scale"
      ],
      "associated_catalogs": [
        "hive_data"
      ],
      "build_version": "1.1.0.0.0",
      "coordinator": {
        "node_type": "bx2.4x16",
        "quantity": 1
      },
      "engine_display_name": "starter",
      "engine_id": "prestissimo511",
      "external_host_name": "your-hostname.apps.your-domain.com",
      "group_id": "prestissimo511",
      "host_name": "1234-xyz456-abc4321.databases.appdomain.cloud",
      "port": 30156,
      "size_config": "starter",
      "status": "running",
      "status_code": 0,
      "tags": [
        "tag1",
        "tag2"
      ],
      "type": "prestissimo",
      "version": "v0.282",
      "worker": {
        "node_type": "bx2.4x16",
        "quantity": 1
      }
    }
  ]
}

Create a new prestissimo engine

POST /prestissimo_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.prestissimo_engine.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*

PrestissimoEnginePrototype

Create Engine Body

Response

Response Body

PrestissimoEngine

EngineDetail

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "actions": [
    "update",
    "delete"
  ],
  "associated_catalogs": [
    "hive_data"
  ],
  "coordinator": {
    "node_type": "worker",
    "quantity": 1
  },
  "created_by": "<username>@<domain>.com",
  "created_on": 163788384993,
  "description": "prestissimo engine for running sql queries",
  "engine_display_name": "sampleEngine",
  "engine_id": "sampleEngine123",
  "external_host_name": "your-hostname.apps.your-domain.com",
  "group_id": "new_group_id",
  "host_name": "your-hostname.apps.your-domain.com",
  "origin": "native",
  "region": "us-south",
  "size_config": "starter",
  "status": "running",
  "status_code": 0,
  "tags": [
    "tag1",
    "tag2"
  ],
  "version": "1.2.0",
  "worker": {
    "node_type": "worker",
    "quantity": 1
  }
}

Get details of one prestissimo engine

GET /prestissimo_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.prestissimo_engine.get

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Response

Response Body

PrestissimoEngine

EngineDetail

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "actions": [
    "update",
    "delete"
  ],
  "associated_catalogs": [
    "hive_data"
  ],
  "coordinator": {
    "node_type": "worker",
    "quantity": 1
  },
  "created_by": "<username>@<domain>.com",
  "created_on": 163788384993,
  "description": "prestissimo engine for running sql queries",
  "engine_display_name": "sampleEngine",
  "engine_id": "sampleEngine123",
  "external_host_name": "your-hostname.apps.your-domain.com",
  "group_id": "new_group_id",
  "host_name": "your-hostname.apps.your-domain.com",
  "origin": "native",
  "region": "us-south",
  "size_config": "starter",
  "status": "running",
  "status_code": 0,
  "tags": [
    "tag1",
    "tag2"
  ],
  "version": "1.2.0",
  "worker": {
    "node_type": "worker",
    "quantity": 1
  }
}

Delete a prestissimo engine

DELETE /prestissimo_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.prestissimo_engine.delete

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Update details of prestissimo engine

PATCH /prestissimo_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.prestissimo_engine.update

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

application/json-patch+jsonPrestissimoEnginePatch

Update prestissimo engine body

curl example

curl -X 'PATCH' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/prestissimo_engines/{engine_id}' -H 'accept: application/json' -H 'Content-Type: application/json-patch+json' -H "AuthInstanceId: {instance id}" -d '{  "description": "updated description for prestissimo engine",  "engine_display_name": "sampleEngine",  "engine_properties": {    "configuration": {      "coordinator": {        "property_1": "property_value",        "property_2": "property_value"      },      "worker": {        "property_1": "property_value",        "property_2": "property_value"      }    },    "catalog": {      "catalog_name": {        "property_1": "property_value",        "property_2": "property_value"      }    },    "velox": {      "property_1": "property_value",      "property_2": "property_value"    }  },  "engine_restart": "force",  "remove_engine_properties": {    "configuration": {      "coordinator": [        "property1", "property2"      ],      "worker": [        "property1", "property2"      ]    },    "catalog": {      "catalog_name": [        "property1", "property2"      ]    },    "velox": [      "property1", "property2"    ]  },  "tags": [    "tag1",    "tag2"  ]}'

Response

Response Body

PrestissimoEngine

EngineDetail

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "actions": [
    "update",
    "delete"
  ],
  "associated_catalogs": [
    "hive_data"
  ],
  "coordinator": {
    "node_type": "worker",
    "quantity": 1
  },
  "created_by": "<username>@<domain>.com",
  "created_on": 163788384993,
  "description": "prestissimo engine for running sql queries",
  "engine_display_name": "sampleEngine",
  "engine_id": "sampleEngine123",
  "external_host_name": "your-hostname.apps.your-domain.com",
  "group_id": "new_group_id",
  "host_name": "your-hostname.apps.your-domain.com",
  "origin": "native",
  "region": "us-south",
  "size_config": "starter",
  "status": "running",
  "status_code": 0,
  "tags": [
    "tag1",
    "tag2"
  ],
  "version": "1.2.0",
  "worker": {
    "node_type": "worker",
    "quantity": 1
  }
}

Get list of all catalogs attached a prestissimo engine

GET /prestissimo_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.prestissimo_engine_catalog.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Response

Response Body

CatalogCollection

GetCatalogs OK

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

{ "catalogs": [] }

Associate one or more catalogs to a prestissimo engine

PUT /prestissimo_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.prestissimo_engine_catalog.add

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Query Parameters

catalog_names
Required*
string
comma separated catalog names

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_,]+$

Response

Response Body

CatalogCollection

GetCatalogs OK

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

{ "catalogs": [] }

Disassociate one or more catalogs from a prestissimo engine

DELETE /prestissimo_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.prestissimo_engine_catalog.remove

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Query Parameters

catalog_names
Required*
string
Catalog id(s) to be stopped, comma separated

Possible values: 1 ≤ length ≤ 1000, Value must match regular expression ^[a-zA-Z0-9\-,_]+

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Get catalog attached to a prestissimo engine

GET /prestissimo_engines/{engine_id}/catalogs/{catalog_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.prestissimo_engine_catalog.get

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+
catalog_id
Required*
string
catalog id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-_]+

Response

Response Body

Catalog

Define the catalog details

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "actions": [
    "view",
    "update",
    "delete"
  ],
  "associated_buckets": [
    "ibm_cos_bucket"
  ],
  "associated_databases": [
    "iceberg_data"
  ],
  "associated_engines": [
    "presto367"
  ],
  "catalog_name": "iceberg_data",
  "catalog_type": "iceberg",
  "created_by": "user@domain.com",
  "created_on": "1700633239",
  "description": "catalog description",
  "hostname": "1234-xyz456-abc4321.lakehouse.dev.appdomain.cloud",
  "port": "31504",
  "sync_exception": [
    "table is corrupted",
    "table metadata not there"
  ],
  "tags": [
    "tag1",
    "tag2"
  ],
  "thrift_uri": "thrift://samplehost-catalog:4354"
}

Pause a running prestissimo engine

POST /prestissimo_engines/{engine_id}/pause

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.prestissimo_engine.pause

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Response

Response Body

SuccessResponse

Response of success

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "message": "Successful message",
  "message_code": "success"
}

Explain a query statement

POST /prestissimo_engines/{engine_id}/query_explain

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.query_explain.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
Engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

ExplainPrototype

Request body

Response

Response Body

ResultPrestissimoExplainStatement

ExplainStatement OK

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

{ "result": "explain statement" }

Return query metrics after query is complete

POST /prestissimo_engines/{engine_id}/query_explain_analyze

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.query_explain_analyze.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
Engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

ExplainAnalyzePrototype

Request body

Response

Response Body

ResultRunPrestissimoExplainAnalyzeStatement

explainAnalyzeStatement OK

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

{ "result": "explain statement" }

Restart an existing prestissimo engine

POST /prestissimo_engines/{engine_id}/restart

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.prestissimo_engine.refresh

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Response

Response Body

SuccessResponse

Response of success

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "message": "Successful message",
  "message_code": "success"
}

Resume a paused prestissimo engine

POST /prestissimo_engines/{engine_id}/resume

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.prestissimo_engine.resume

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Response

Response Body

SuccessResponse

Response of success

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "message": "Successful message",
  "message_code": "success"
}

Scale an existing prestissimo engine

POST /prestissimo_engines/{engine_id}/scale

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.prestissimo_engine.scale

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

PrestissimoEngineScale

Scale Engine Body

Response

Response Body

SuccessResponse

Response of success

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "message": "Successful message",
  "message_code": "success"
}

Get list of all presto engines

GET /presto_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.presto_engine.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

curl example

curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/presto_engines"

Response

Response Body

PrestoEngineCollection

List Presto engines

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "presto_engines": [
    {
      "actions": [
        "view",
        "use",
        "update",
        "select",
        "access_ui",
        "associate",
        "disassociate",
        "restart",
        "pause",
        "resume",
        "grant",
        "revoke",
        "delete",
        "create",
        "scale"
      ],
      "associated_catalogs": [
        "iceberg_data",
        "hive_data"
      ],
      "build_version": "1.1.0.0.0",
      "coordinator": {
        "node_type": "bx2.4x16",
        "quantity": 1
      },
      "engine_display_name": "starter",
      "engine_id": "presto511",
      "external_host_name": "your-hostname.apps.your-domain.com",
      "group_id": "presto511",
      "host_name": "1234-xyz456-abc4321.databases.appdomain.cloud",
      "origin": "native",
      "port": 30156,
      "size_config": "starter",
      "status": "running",
      "status_code": 0,
      "tags": [
        "tag1",
        "tag2"
      ],
      "type": "presto",
      "version": "v0.282",
      "worker": {
        "node_type": "bx2.4x16",
        "quantity": 1
      }
    }
  ]
}

Create a new presto engine

POST /presto_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.presto_engine.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*

PrestoEnginePrototype

Create Engine Body

curl example

curl -X 'POST' \'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/presto_engines' \-H 'accept: application/json' \-H 'Content-Type: application/json' \-d '{  "associated_catalogs": [    "iceberg_data",    "hive_data"  ],  "description": "presto engine description",  "engine_details": {    "api_key": "<api_key>",    "connection_string": "1.2.3.4",    "coordinator": {      "node_type": "worker",      "quantity": 0    },    "instance_id": "instance_id",    "managed_by": "fully/self",    "size_config": "starter",    "worker": {      "node_type": "worker",      "quantity": 0    }  },  "engine_display_name": "sampleEngine",  "first_time_use": true,  "origin": "native",  "region": "us-south",  "tags": [    "tag1",    "tag2"  ],  "type": "presto",  "version": "1.2.3"}'

Response

Response Body

PrestoEngine

EngineDetail

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "actions": [
    "view",
    "use",
    "update",
    "select",
    "access_ui",
    "associate",
    "disassociate",
    "restart",
    "pause",
    "resume",
    "grant",
    "revoke",
    "delete",
    "create",
    "scale"
  ],
  "associated_catalogs": [
    "iceberg_data",
    "hive_data"
  ],
  "build_version": "1.1.0.0.0",
  "coordinator": {
    "node_type": "bx2.4x16",
    "quantity": 1
  },
  "engine_display_name": "starter",
  "engine_id": "presto511",
  "external_host_name": "your-hostname.apps.your-domain.com",
  "group_id": "presto511",
  "host_name": "1234-xyz456-abc4321.databases.appdomain.cloud",
  "origin": "native",
  "port": 30156,
  "size_config": "starter",
  "status": "running",
  "status_code": 0,
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "presto",
  "version": "v0.282",
  "worker": {
    "node_type": "bx2.4x16",
    "quantity": 1
  }
}

Get details of one presto engine

GET /presto_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.presto_engine.get

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'GET' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}' -H 'accept: application/json' -H "AuthInstanceId: {instance id}" 

Response

Response Body

PrestoEngine

EngineDetail

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "actions": [
    "view",
    "use",
    "update",
    "select",
    "access_ui",
    "associate",
    "disassociate",
    "restart",
    "pause",
    "resume",
    "grant",
    "revoke",
    "delete",
    "create",
    "scale"
  ],
  "associated_catalogs": [
    "iceberg_data",
    "hive_data"
  ],
  "build_version": "1.1.0.0.0",
  "coordinator": {
    "node_type": "bx2.4x16",
    "quantity": 1
  },
  "engine_display_name": "starter",
  "engine_id": "presto511",
  "external_host_name": "your-hostname.apps.your-domain.com",
  "group_id": "presto511",
  "host_name": "1234-xyz456-abc4321.databases.appdomain.cloud",
  "origin": "native",
  "port": 30156,
  "size_config": "starter",
  "status": "running",
  "status_code": 0,
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "presto",
  "version": "v0.282",
  "worker": {
    "node_type": "bx2.4x16",
    "quantity": 1
  }
}

Delete a presto engine

DELETE /presto_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.presto_engine.delete

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'DELETE' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}' -H 'accept: */*' -H "AuthInstanceId: {instance id}"

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Update details of presto engine

PATCH /presto_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.presto_engine.update

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

application/json-patch+jsonPrestoEnginePatch

Update Engine Body

curl example

curl -X 'PATCH' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}' -H 'accept: application/json' -H 'Content-Type: application/json-patch+json' -H "AuthInstanceId: {instance id}" -d '{  "description": "updated description for presto engine",  "engine_display_name": "sampleEngine",  "engine_properties": {    "configuration": {      "coordinator": {        "property1": "value1"      },      "worker": {        "property1": "value1"      }    },    "jvm": {      "coordinator": {        "property1": "value1"      },      "worker": {        "property1": "value1"      }    },    "catalog": {      "catalog_name": {        "property1": "value1"      }    },    "global": {      "enable-mixed-case-support": "true"    }  },  "engine_restart": "force",  "remove_engine_properties": {    "configuration": {      "coordinator": [      ],      "worker": [      ]    },    "jvm": {      "coordinator": [      ],      "worker": [      ]    },  },  "tags": [    "tag1",    "tag2"  ]}'

Response

Response Body

PrestoEngine

EngineDetail

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "actions": [
    "view",
    "use",
    "update",
    "select",
    "access_ui",
    "associate",
    "disassociate",
    "restart",
    "pause",
    "resume",
    "grant",
    "revoke",
    "delete",
    "create",
    "scale"
  ],
  "associated_catalogs": [
    "iceberg_data",
    "hive_data"
  ],
  "build_version": "1.1.0.0.0",
  "coordinator": {
    "node_type": "bx2.4x16",
    "quantity": 1
  },
  "engine_display_name": "starter",
  "engine_id": "presto511",
  "external_host_name": "your-hostname.apps.your-domain.com",
  "group_id": "presto511",
  "host_name": "1234-xyz456-abc4321.databases.appdomain.cloud",
  "origin": "native",
  "port": 30156,
  "size_config": "starter",
  "status": "running",
  "status_code": 0,
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "presto",
  "version": "v0.282",
  "worker": {
    "node_type": "bx2.4x16",
    "quantity": 1
  }
}

Get list of all catalogs attached to a presto engine

GET /presto_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.presto_engine_catalog.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'GET' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/catalogs' -H 'accept: application/json' -H "AuthInstanceId: {instance id}"

Response

Response Body

CatalogCollection

GetCatalogs OK

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

{ "catalogs": [] }

Associate one or more catalogs to a presto engine

PUT /presto_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.presto_engine_catalog.add

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Query Parameters

catalog_names
Required*
string
comma separated catalog names

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_,]+$

curl example

curl -X 'PUT' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/catalogs?catalog_names=catalog_name1' -H 'accept: application/json' -H "AuthInstanceId: {instance id}"

Response

Response Body

CatalogCollection

GetCatalogs OK

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

{ "catalogs": [] }

Disassociate one or more catalogs from a presto engine

DELETE /presto_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.presto_engine_catalog.remove

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Query Parameters

catalog_names
Required*
string
Catalog id(s) to be stopped, comma separated

Possible values: 1 ≤ length ≤ 1000, Value must match regular expression ^[a-zA-Z0-9\-_,]+

curl example

curl -X 'DELETE' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/catalogs?catalog_names=catalog_name1' -H 'accept: */*' -H "AuthInstanceId: {instance id}"

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Get catalog attached to presto engine

GET /presto_engines/{engine_id}/catalogs/{catalog_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.presto_engine_catalog.get

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+
catalog_id
Required*
string
catalog id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-_]+

curl example

curl -X 'GET' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/catalogs/{catalog_id}' -H 'accept: application/json' -H "AuthInstanceId: {instance id}"

Response

Response Body

Catalog

Define the catalog details

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "actions": [
    "view",
    "update",
    "delete"
  ],
  "associated_buckets": [
    "ibm_cos_bucket"
  ],
  "associated_databases": [
    "iceberg_data"
  ],
  "associated_engines": [
    "presto367"
  ],
  "catalog_name": "iceberg_data",
  "catalog_type": "iceberg",
  "created_by": "user@domain.com",
  "created_on": "1700633239",
  "description": "catalog description",
  "hostname": "1234-xyz456-abc4321.lakehouse.dev.appdomain.cloud",
  "port": "31504",
  "sync_exception": [
    "table is corrupted",
    "table metadata not there"
  ],
  "tags": [
    "tag1",
    "tag2"
  ],
  "thrift_uri": "thrift://samplehost-catalog:4354"
}

Pause a running presto engine

POST /presto_engines/{engine_id}/pause

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.presto_engine.pause

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'POST' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/pause' -H 'accept: application/json' -H "AuthInstanceId: {instance id}" -d ''

Response

Response Body

CreateEnginePauseCreatedBody

Pause

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "response": {
    "message": "pause presto engine",
    "message_code": "success"
  }
}

Explain a query statement

POST /presto_engines/{engine_id}/query_explain

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.query_explain.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
Engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

ExplainPrototype

Request body

curl example

curl -X 'POST' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/query_explain' -H 'accept: application/json' -H 'Content-Type: application/json' -H "AuthInstanceId: {instance id}" -d '{  "format": "json",  "statement": "show schemas in catalog_name",  "type": "io"}'

Response

Response Body

RunExplainStatementOKBody

ExplainStatement OK

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "response": {
    "message": "explain statement",
    "message_code": "success"
  },
  "result": "explain statement"
}

Return query metrics after query is complete

POST /presto_engines/{engine_id}/query_explain_analyze

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.query_explain_analyze.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
Engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

ExplainAnalyzePrototype

Request body

curl example

curl -X 'POST' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/query_explain_analyze' -H 'accept: application/json' -H 'Content-Type: application/json' -H "AuthInstanceId: {instance id}" -d '{  "statement": "show schemas in catalog_name",  "verbose": true}'

Response

Response Body

RunExplainAnalyzeStatementOKBody

explainAnalyzeStatement OK

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "response": {
    "message": "explain presto analyze",
    "message_code": "success"
  },
  "result": "explain statement"
}

Restart an existing presto engine

POST /presto_engines/{engine_id}/restart

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.presto_engine.refresh

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'POST' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/restart' -H 'accept: application/json' -H "AuthInstanceId: {instance id}" -d ''

Response

Response Body

CreateEngineRestartCreatedBody

restart engine

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "response": {
    "message": "Restart presto engine",
    "message_code": "success"
  }
}

Resume a paused presto engine

POST /presto_engines/{engine_id}/resume

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.presto_engine.resume

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'POST' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/resume' -H 'accept: application/json' -H "AuthInstanceId: {instance id}" -d ''

Response

Response Body

CreateEngineResumeCreatedBody

resume

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "response": {
    "message": "resume presto engine",
    "message_code": "success"
  }
}

Scale an existing presto engine

POST /presto_engines/{engine_id}/scale

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.presto_engine.scale

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

PrestoEngineScale

Scale Engine Body

curl example

curl -X 'POST' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/scale' -H 'accept: application/json' -H 'Content-Type: application/json' -H "AuthInstanceId: {instance id}" -d '{  "coordinator": {    "node_type": "worker",    "quantity": 0  },  "worker": {    "node_type": "worker",    "quantity": 0  }}'

Response

Response Body

CreateEngineScaleCreatedBody

scale engine

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "response": {
    "message": "scale presto engine",
    "message_code": "success"
  }
}

List all spark engines

GET /spark_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

curl example

curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines"

Response

Response Body

SparkEngineCollection

List spark engines

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "spark_engines": [
    {
      "actions": [
        "action1",
        "action2",
        "action3"
      ],
      "associated_catalogs": [],
      "created_by": "user@example.com",
      "created_on": 1234567890,
      "description": "Sample Description",
      "engine_details": {
        "history_server_endpoint": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_api/v1",
        "spark_access_endpoint": "https://console.example.com/analytics-engine/details/spark-1717414394213457",
        "spark_jobs_v4_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/spark_applications",
        "spark_kernel_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/jkg/api/kernels",
        "view_history_server": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_ui",
        "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/abcd-efgh-ijkl-mnop/spark_engines/spark119/applications"
      },
      "engine_display_name": "sample-external-spark-co-located",
      "engine_id": "spark456",
      "origin": "external",
      "status": "REGISTERED",
      "tags": [
        "tag1",
        "tag2"
      ],
      "type": "spark"
    },
    {
      "actions": [
        "action1",
        "action2",
        "action3"
      ],
      "associated_catalogs": [
        "iceberg_data",
        "hudi_catalog",
        "delta_lake"
      ],
      "created_by": "user@example.com",
      "created_on": 1234567890,
      "description": "Sample Description",
      "engine_details": {
        "default_config": {
          "config1": "value1",
          "config2": "value2"
        },
        "default_version": "3.3",
        "endpoints": {
          "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/spark_engines/spark123/applications",
          "wxd_engine_endpoint": "https://example.com/lakehouse/api/v2/spark_engines/spark123",
          "wxd_history_server_endpoint": "https://example.com/lakehouse/api/v2/spark_engines/spark123/history_server",
          "wxd_history_server_ui_endpoint": "https://example.com/lakehouse/api/v2/1234567890123456/spark_engines/spark123/history_server/ui"
        },
        "engine_home_volume": "cpd-instance::volume-name"
      },
      "engine_display_name": "sample-native-spark",
      "engine_id": "spark123",
      "origin": "native",
      "status": "running",
      "tags": [],
      "type": "spark"
    }
  ]
}

Create a new spark engine

POST /spark_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*

SparkEnginePrototype

Create spark engine Body

Examples:

NativeSparkEngineExistingVolume

NativeSparkEngineNewVolume

ExternalSparkEngineCoLocated

ExternalSparkEngineFullyManaged

ExternalSparkEngineSelfManaged

Native Spark Engine Existing Volume

curl -X POST -H "content-type: application/json" -H "accept: application/json" -d '{ "description": "","engine_details": {"default_version": "3.3","engine_home_bucket_name": "4fec0f8b-888a-4c16-8f38-250c8499e6ce-customer"},"engine_display_name": "sample-native-spark-existing-volume","associated_catalogs": ["iceberg_data"],"origin": "native","type": "spark"}' "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines"

Native Spark Engine New Volume

curl -X POST -H "content-type: application/json" -H "accept: application/json" -d '{"description": "","engine_details": {"default_version": "3.3","engine_home_volume_storage_size": "5Gi","engine_home_volume_name": "volume-name","engine_home_volume_storage_class": "nfs-client"},"engine_display_name": "sample-native-spark-new-volume","associated_catalogs": ["iceberg_data","hive_data"],"origin": "native","type": "spark"}' "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines"

External Spark Engine CoLocated

curl -X POST -H "content-type: application/json" -H "accept: application/json" -d '{"description": "","engine_details": {"instance_id": "1716437041193635"},"engine_display_name": "sample-external-spark-co-located","origin": "discover","type": "spark"}'  "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines"

External Spark Engine Fully Managed

curl -X POST -H "content-type: application/json" -H "accept: application/json" -d '{"description": "","engine_details": {"connection_string": "https://<host>/<version>/analytics_engines/<instance_id>","api_key": "api_Key","managed_by": "fully"},"engine_display_name": "sample-external-spark-fully-managed","origin": "external","type": "spark"}'  "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines"

External Spark Engine Self Managed

curl -X POST -H "content-type: application/json" -H "accept: application/json" -d '{"description": "","engine_details": {"connection_string": "hhttps://<host>/<version>/analytics_engines/<instance_id>","api_key": "zen_api_Key","managed_by": "self"},"engine_display_name": "sample-external-spark-self-managed","origin": "external","type": "spark"}'  "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines"

Response

Response Body

SparkEngine

EngineDetail

Status Code

202
Accepted
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 202

{
  "actions": [],
  "associated_catalogs": [
    "iceberg_data"
  ],
  "engine_id": "spark123",
  "origin": "native",
  "status": "runnung",
  "tags": [
    "tag1",
    "tag2"
  ]
}

Status 202

{
  "actions": [],
  "associated_catalogs": [],
  "engine_id": "spark530",
  "origin": "discover",
  "status": "registered",
  "tags": [
    "tag1",
    "tag2"
  ]
}

Status 202

{
  "actions": [],
  "associated_catalogs": [],
  "engine_id": "spark810",
  "origin": "external",
  "status": "",
  "tags": [
    "tag1",
    "tag2"
  ]
}

Status 202

{
  "actions": [],
  "associated_catalogs": [],
  "engine_id": "spark728",
  "origin": "external",
  "status": "registered",
  "tags": [
    "tag1",
    "tag2"
  ]
}

Get spark engine by ID

GET /spark_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine.get

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X GET -H "accept: */*" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}"

Response

Response Body

SparkEngine

EngineDetail

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "actions": [
    "action1",
    "action2",
    "action3"
  ],
  "associated_catalogs": [
    "iceberg_data",
    "hudi_catalog",
    "delta_lake"
  ],
  "created_by": "user@example.com",
  "created_on": 1234567890,
  "description": "Sample Description",
  "engine_details": {
    "default_config": {
      "config1": "value1",
      "config2": "value2"
    },
    "default_version": "3.3",
    "endpoints": {
      "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/spark_engines/spark123/applications",
      "wxd_engine_endpoint": "https://example.com/lakehouse/api/v2/spark_engines/spark123",
      "wxd_history_server_endpoint": "https://example.com/lakehouse/api/v2/spark_engines/spark123/history_server",
      "wxd_history_server_ui_endpoint": "https://example.com/lakehouse/api/v2/1234567890123456/spark_engines/spark123/history_server/ui"
    },
    "engine_home_volume": "cpd-instance::volume-name"
  },
  "engine_display_name": "sample-narive-spark",
  "engine_id": "spark123",
  "origin": "native",
  "status": "running",
  "tags": [],
  "type": "spark"
}

Status 200

{
  "actions": [
    "action1",
    "action2",
    "action3"
  ],
  "associated_catalogs": [],
  "created_by": "user@example.com",
  "created_on": 1234567890,
  "description": "Sample Description",
  "engine_details": {
    "history_server_endpoint": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_api/v1",
    "spark_access_endpoint": "https://console.example.com/analytics-engine/details/spark-1234567890123456",
    "spark_jobs_v4_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/spark_applications",
    "spark_kernel_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/jkg/api/kernels",
    "view_history_server": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_ui",
    "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/abcd-efgh-ijkl-mnop/spark_engines/spark119/applications"
  },
  "engine_display_name": "sample-external-spark-co-located",
  "engine_id": "spark456",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "spark"
}

Status 200

{
  "actions": [
    "action1",
    "action2",
    "action3"
  ],
  "associated_catalogs": [],
  "created_by": "user@example.com",
  "created_on": 1234567890,
  "description": "Sample Description",
  "engine_details": {
    "applications_api": "https://api.example.com/v3/analytics_engines/abcd-efgh-ijkl-mnop/applications",
    "history_server_endpoint": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_api/v1",
    "view_history_server": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_ui",
    "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/abcd-efgh-ijkl-mnop/spark_engines/spark119/applications"
  },
  "engine_display_name": "sample-external-spark-fully-managed",
  "engine_id": "spark456",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "spark"
}

Status 200

{
  "actions": [
    "action1",
    "action2",
    "action3"
  ],
  "associated_catalogs": [],
  "created_by": "user@example.com",
  "created_on": 1234567890,
  "description": "Sample Description",
  "engine_details": {
    "connection_string": "https://api.example.com/v3/analytics_engines/abcd-efgh-ijkl-mnop",
    "endpoints": {
      "history_server_endpoint": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_api/v1",
      "spark_access_endpoint": "https://console.example.com/analytics-engine/details/spark-1234567890123456",
      "spark_jobs_v4_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/spark_applications",
      "spark_kernel_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/jkg/api/kernels",
      "view_history_server": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_ui",
      "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/abcd-efgh-ijkl-mnop/spark_engines/spark119/applications"
    }
  },
  "engine_display_name": "sample-external-spark-self-managed",
  "engine_id": "spark456",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "spark"
}

Delete a spark engine

DELETE /spark_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine.delete

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X DELETE -H "accept: */*" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}"

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Update details of spark engine

PATCH /spark_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine.update

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

application/merge-patch+jsonUpdateSparkEngineBody

Update Engine Body

Examples:

NativeSparkEngine

ExternalSparkEngine

Native Spark Engine

curl -X PATCH -H "content-type: application/json" -H "accept: application/json" -d '{ "description": "Updated Description", "engine_details": { "default_config": { "config1": "value1", "config2": "value2" }, "default_version": "3.4" }, "engine_display_name": "Updated Display Name", "tags": [ "tag1", "tag2" ] }' "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}"

External Spark Engine

curl -X PATCH -H "content-type: application/json" -H "accept: application/json" -d '{ "description": "Updated description", "engine_display_name": "Updated Display Name", "tags": [ "tag1" ] }' "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}"

Response

Response Body

SparkEngine

EngineDetail

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "actions": [
    "action1",
    "action2",
    "action3"
  ],
  "associated_catalogs": [
    "iceberg_data",
    "hudi_catalog",
    "delta_lake"
  ],
  "created_by": "user@example.com",
  "created_on": 1234567890,
  "description": "Sample Description",
  "engine_details": {
    "default_config": {
      "config1": "value1",
      "config2": "value2"
    },
    "default_version": "3.3",
    "endpoints": {
      "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/spark_engines/spark123/applications",
      "wxd_engine_endpoint": "https://example.com/lakehouse/api/v2/spark_engines/spark123",
      "wxd_history_server_endpoint": "https://example.com/lakehouse/api/v2/spark_engines/spark123/history_server",
      "wxd_history_server_ui_endpoint": "https://example.com/lakehouse/api/v2/1234567890123456/spark_engines/spark123/history_server/ui"
    },
    "engine_home_volume": "cpd-instance::volume-name"
  },
  "engine_display_name": "sample-narive-spark",
  "engine_id": "spark123",
  "origin": "native",
  "status": "running",
  "tags": [],
  "type": "spark"
}

Status 200

{
  "actions": [
    "action1",
    "action2",
    "action3"
  ],
  "associated_catalogs": [],
  "created_by": "user@example.com",
  "created_on": 1234567890,
  "description": "Sample Description",
  "engine_details": {
    "history_server_endpoint": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_api/v1",
    "spark_access_endpoint": "https://console.example.com/analytics-engine/details/spark-1234567890123456",
    "spark_jobs_v4_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/spark_applications",
    "spark_kernel_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/jkg/api/kernels",
    "view_history_server": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_ui",
    "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/abcd-efgh-ijkl-mnop/spark_engines/spark119/applications"
  },
  "engine_display_name": "sample-external-spark-co-located",
  "engine_id": "spark456",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "spark"
}

Status 200

{
  "actions": [
    "action1",
    "action2",
    "action3"
  ],
  "associated_catalogs": [],
  "created_by": "user@example.com",
  "created_on": 1234567890,
  "description": "Sample Description",
  "engine_details": {
    "applications_api": "https://api.example.com/v3/analytics_engines/abcd-efgh-ijkl-mnop/applications",
    "history_server_endpoint": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_api/v1",
    "view_history_server": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_ui",
    "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/abcd-efgh-ijkl-mnop/spark_engines/spark119/applications"
  },
  "engine_display_name": "sample-external-spark-fully-managed",
  "engine_id": "spark456",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "spark"
}

Status 200

{
  "actions": [
    "action1",
    "action2",
    "action3"
  ],
  "associated_catalogs": [],
  "created_by": "user@example.com",
  "created_on": 1234567890,
  "description": "Sample Description",
  "engine_details": {
    "connection_string": "https://api.example.com/v3/analytics_engines/abcd-efgh-ijkl-mnop",
    "endpoints": {
      "history_server_endpoint": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_api/v1",
      "spark_access_endpoint": "https://console.example.com/analytics-engine/details/spark-1234567890123456",
      "spark_jobs_v4_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/spark_applications",
      "spark_kernel_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/jkg/api/kernels",
      "view_history_server": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_ui",
      "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/abcd-efgh-ijkl-mnop/spark_engines/spark119/applications"
    }
  },
  "engine_display_name": "sample-external-spark-self-managed",
  "engine_id": "spark456",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "spark"
}

List all applications in a spark engine

GET /spark_engines/{engine_id}/applications

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine_application.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Query Parameters

state
string[]
state

Possible values: 0 ≤ number of items ≤ 100, 1 ≤ length ≤ 128, Value must match regular expression .*

curl example

curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/applications"

Response

Response Body

SparkEngineApplicationStatusCollection

Engine Application Detail

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "applications": [
    {
      "application_id": "93b1a765-e3bc-4972-acc4-5b3d1a0f5d62",
      "auto_termination_time": "2024-05-15T18:11:54.615Z",
      "id": "93b1a765-e3bc-4972-acc4-5b3d1a0f5d62",
      "spark_application_id": "app-20240415181154-0000",
      "spark_application_name": "auto-scale-test",
      "spark_version": "3.4",
      "start_time": "2024-04-15T18:11:54.615Z",
      "state": "running",
      "submission_time": "2024-04-15T18:11:21.458Z"
    },
    {
      "application_id": "b2d3f4g5-h6i7-j8k9-l0m1-n2o3p4q5r6s7",
      "auto_termination_time": "2025-06-16T19:12:55.716Z",
      "id": "b2d3f4g5-h6i7-j8k9-l0m1-n2o3p4q5r6s7",
      "spark_application_id": "app-20250516191255-0001",
      "spark_application_name": "data-processing-test",
      "spark_version": "3.3",
      "start_time": "2025-05-16T19:12:55.716Z",
      "state": "finished",
      "submission_time": "2025-05-16T19:12:22.559Z"
    }
  ]
}

Status 200

{
  "applications": [
    {
      "application_id": "abcd-efgh-5806ab96-a5ba-4488-871b-162e13f6bf90",
      "id": "abcd-efgh-5806ab96-a5ba-4488-871b-162e13f6bf90",
      "runtime": {
        "spark_version": "3.3"
      },
      "state": "accepted",
      "submission_time": "2024-04-15T18:13:17.507Z"
    },
    {
      "application_id": "abcd-efgh-0969c82b-2980-48f4-8896-800e311a4696",
      "auto_termination_time": "2024-03-25T15:57:06.762Z",
      "end_time": "2024-03-22T15:57:24.703Z",
      "finish_time": "2024-03-22T15:57:24.703Z",
      "id": "abcd-efgh-0969c82b-2980-48f4-8896-800e311a4696",
      "runtime": {
        "spark_version": "3.3"
      },
      "spark_application_id": "app-20240322155706-0000",
      "spark_application_name": "PythonWordCount",
      "start_time": "2024-03-22T15:57:06.762Z",
      "state": "finished",
      "submission_time": "2024-03-22T15:56:41.400Z"
    }
  ]
}

Submit engine applications

POST /spark_engines/{engine_id}/applications

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine_application.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Query Parameters

state
string[]
state

Possible values: 0 ≤ number of items ≤ 100, 1 ≤ length ≤ 128, Value must match regular expression .*

Request Body

Required*

SparkEngineApplicationStatusPrototype

Create spark application parameters

Examples:

View

curl example

curl -X POST -H "content-type: application/json" -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -d '{ "application_details": { "application": "s3://mybucket/wordcount.py", "arguments": [ "people.txt" ], "conf": { "additionalProp1": "key:value", "additionalProp2": "key:value", "additionalProp3": "key:value" }, "env": { "additionalProp1": "key:value", "additionalProp2": "key:value", "additionalProp3": "key:value" }, "name": "SparkApplicaton1" }, "job_endpoint": "<host>/v4/analytics_engines/c7b3fccf-badb-46b0-b1ef-9b3154424021/engine_applications", "service_instance_id": "3FGeNUZqcdu6rOFbaHhGBoNdCHj98UoHd6nUFqkfsKhgacYD8ImM6iSA_F-TjHUdEZuVsoQTmOn", "type": "iae" }' "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/applications"

Response

Response Body

SparkEngineApplicationStatus

Engine Application Status

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "id": "2ad2af4b-9747-4652-8e6e-d0a91599f9a2",
  "state": "accepted"
}

Stop a running spark application

DELETE /spark_engines/{engine_id}/applications

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine_application.delete

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Query Parameters

application_id
Required*
string
Application id(s) to be stopped, comma separated

Possible values: 1 ≤ length ≤ 1000, Value must match regular expression ^[a-zA-Z0-9\-]+
state
string[]
state

Possible values: 0 ≤ number of items ≤ 100, 1 ≤ length ≤ 128, Value must match regular expression .*

curl example

curl -X DELETE -H "accept: */*" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/applications?application_id={application_id}"

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Get status of spark application

GET /spark_engines/{engine_id}/applications/{application_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine_application.get

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+
application_id
Required*
string
Application id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance_id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/applications/{application_id}"

Response

Response Body

SparkEngineApplicationStatus

Engine Application Status

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "application_details": {
    "application": "/opt/ibm/spark/examples/src/main/python/wordcount.py",
    "conf": {
      "ae.spark.executor.count": "1",
      "spark.app.name": "MyJob",
      "spark.driver.cores": "1",
      "spark.driver.memory": "4G",
      "spark.eventLog.enabled": "true",
      "spark.executor.cores": "1",
      "spark.executor.memory": "4G"
    },
    "env": {
      "SAMPLE_ENV_KEY": "SAMPLE_VALUE"
    },
    "spark_version": "3.3"
  },
  "application_id": "ff053468-9d89-468e-a5af-bd96b88c5bf5",
  "auto_termination_time": "2024-05-16T06:56:51.469Z",
  "end_time": "2024-04-16T06:57:19.912Z",
  "finish_time": "2024-04-16T06:57:19.912Z",
  "id": "ff053468-9d89-468e-a5af-bd96b88c5bf5",
  "return_code": "0",
  "spark_application_id": "app-20240416065651-0000",
  "spark_application_name": "PythonWordCount",
  "spark_version": "3.3",
  "start_time": "2024-04-16T06:56:51.469Z",
  "state": "finished",
  "submission_time": "2024-04-16T06:56:22.909Z"
}

Get list of all catalogs attached to a spark engine

GET /spark_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine_catalog.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'GET' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/catalogs' -H 'accept: application/json' -H "AuthInstanceId: {instance id}"

Response

Response Body

CatalogCollection

GetCatalogs OK

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "catalogs": [
    {
      "actions": [
        "action1",
        "action2"
      ],
      "associated_buckets": [],
      "associated_databases": [],
      "associated_engines": [],
      "catalog_name": "iceberg_data",
      "sync_exception": [],
      "tags": [
        "tag1",
        "tag2"
      ]
    },
    {
      "actions": [
        "action3",
        "action4"
      ],
      "associated_buckets": [],
      "associated_databases": [],
      "associated_engines": [],
      "catalog_name": "hudi",
      "sync_exception": [],
      "tags": [
        "tag3",
        "tag4"
      ]
    },
    {
      "actions": [
        "action5",
        "action6"
      ],
      "associated_buckets": [],
      "associated_databases": [],
      "associated_engines": [],
      "catalog_name": "delta_lake",
      "sync_exception": [],
      "tags": [
        "tag5",
        "tag6"
      ]
    }
  ]
}

Associate one or more catalogs to a spark engine

PUT /spark_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine_catalog.add

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Query Parameters

catalog_names
Required*
string
comma separated catalog names

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_,]+$

curl example

curl -X 'PUT' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/catalogs?catalog_names=catalog_name1' -H 'accept: application/json' -H "AuthInstanceId: {instance id}"

Response

Response Body

CatalogCollection

GetCatalogs OK

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "catalogs": [
    {
      "actions": [
        "action1",
        "action2"
      ],
      "catalog_name": "iceberg_data",
      "sync_exception": [],
      "tags": [
        "tag1",
        "tag2"
      ]
    }
  ]
}

Disassociate one or more catalogs from a spark engine

DELETE /spark_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine_catalog.remove

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Query Parameters

catalog_names
Required*
string
Catalog id(s) to be stopped, comma separated

Possible values: 1 ≤ length ≤ 1000, Value must match regular expression ^[a-zA-Z0-9\-_,]+

curl example

curl -X 'DELETE' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/catalogs?catalog_names=catalog_name1' -H 'accept: */*' -H "AuthInstanceId: {instance id}"

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Get catalog attached to spark engine

GET /spark_engines/{engine_id}/catalogs/{catalog_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine_catalog.get

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+
catalog_id
Required*
string
catalog id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-_]+

curl example

curl -X 'GET' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/catalogs/{catalog_id}' -H 'accept: application/json' -H "AuthInstanceId: {instance id}"

Response

Response Body

Catalog

Define the catalog details

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "actions": [],
  "associated_buckets": [
    "d501dc2b-05db-4d93-9aed-edf93c7bb0f6-customer"
  ],
  "associated_databases": [],
  "associated_engines": [
    "presto459",
    "spark422"
  ],
  "catalog_name": "iceberg_data",
  "catalog_type": "iceberg",
  "created_by": "user@example.com",
  "created_on": "1713155437",
  "description": "icebarg_data",
  "hostname": "hostname",
  "port": "<port>",
  "sync_exception": [],
  "tags": [],
  "thrift_uri": "thrift://<hostname>:<port>"
}

Get spark history server

GET /spark_engines/{engine_id}/history_server

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine.history_server

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/history_server"

Response

Response Body

SparkHistoryServer

Native spark history server

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "auto_termination_time": "2022-02-24T07:37:47Z",
  "cores": "1",
  "memory": "4G",
  "start_time": "2022-02-21T07:37:47Z",
  "state": "started"
}

Start spark history server

POST /spark_engines/{engine_id}/history_server

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine.history_server

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

StartSparkEngineHistoryServerParamsBody

Start history server parameter

curl example

curl -X POST -H "content-type: application/json" -H "AuthInstanceId: {instance id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/history_server"

Response

Response Body

SparkHistoryServer

Native spark history server

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "auto_termination_time": "2022-02-24T07:37:47Z",
  "cores": "1",
  "memory": "4G",
  "start_time": "2022-02-21T07:37:47Z",
  "state": "started"
}

Stop spark history server

DELETE /spark_engines/{engine_id}/history_server

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine.history_server

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X DELETE -H "content-type: application/json" -H "AuthInstanceId: {instance id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/history_server"

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Pause engine

POST /spark_engines/{engine_id}/pause

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine.pause

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X POST -H "content-type: application/json" -H "AuthInstanceId: {instance id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/pause"

Response

Response Body

SuccessResponse

Response of success

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "message": "Successful message",
  "message_code": "success"
}

Resume engine

POST /spark_engines/{engine_id}/resume

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine.resume

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X POST -H "content-type: application/json" -H "AuthInstanceId: {instance id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/resume"

Response

Response Body

SuccessResponse

Response of success

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "message": "Successful message",
  "message_code": "success"
}

Scale Saprk engine

POST /spark_engines/{engine_id}/scale

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_engine.scale

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

CreateSparkEngineScaleParamsBody

Scale spark application parameters

curl example

curl -X POST -H "content-type: application/json" -H "AuthInstanceId: {instance id}" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/scale"-d '{  "number_of_nodes": 2}'

Response

Response Body

SuccessResponse

Response of success

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "message": "Successful message",
  "message_code": "success"
}

List spark version

GET /spark_versions

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.spark_versions.read

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Response

Response Body

ListSparkVersionsOKBody

List spark version

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "response": {
    "message": "List spark versions",
    "message_code": "success"
  },
  "spark_versions": [
    {
      "display_name": "3.3"
    }
  ]
}

Verify lhconsole server is up and running

GET /ready

Request

No Request Parameters

This method does not accept any request parameters.

curl example

curl -X GET -H "accept: application/json" "https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/ready"

Response

Response Body

SuccessResponse

Response of success

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "message": "Successful message",
  "message_code": "success"
}

List all registered catalogs

GET /catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.catalog.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

curl example

curl -X 'GET' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/catalogs' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

Response

Response Body

CatalogCollection

GetCatalogs OK

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

{ "catalogs": [] }

Get catalog properties of a catalog identified by catalog_id

GET /catalogs/{catalog_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.catalog.get

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

catalog_id
Required*
string
catalog ID

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-_]+

curl example

curl -X 'GET' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

Response

Response Body

Catalog

Define the catalog details

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "actions": [
    "view",
    "update",
    "delete"
  ],
  "associated_buckets": [
    "ibm_cos_bucket"
  ],
  "associated_databases": [
    "iceberg_data"
  ],
  "associated_engines": [
    "presto367"
  ],
  "catalog_name": "iceberg_data",
  "catalog_type": "iceberg",
  "created_by": "user@domain.com",
  "created_on": "1700633239",
  "description": "catalog description",
  "hostname": "1234-xyz456-abc4321.lakehouse.dev.appdomain.cloud",
  "port": "31504",
  "sync_exception": [
    "table is corrupted",
    "table metadata not there"
  ],
  "tags": [
    "tag1",
    "tag2"
  ],
  "thrift_uri": "thrift://samplehost-catalog:4354"
}

List all schemas in catalog

GET /catalogs/{catalog_id}/schemas

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.schema.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

catalog_id
Required*
string
Catalog name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+

Query Parameters

engine_id
Required*
string
Engine name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'GET' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

Response

Response Body

ListSchemasOKBody

GetSchemas OK

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "response": {
    "message": "create schema",
    "message_code": "success"
  },
  "schemas": [
    "schema1",
    "schema2"
  ]
}

Create a new schema

POST /catalogs/{catalog_id}/schemas

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.schema.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

catalog_id
Required*
string
Catalog name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+

Query Parameters

engine_id
Required*
string
Engine name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

CreateSchemaBody

Request body

curl example

curl -X 'POST' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}' -H 'Content-Type: application/json' -d '{
  "bucket_name": "sample-bucket", 
  "custom_path": "sample-path", 
  "schema_name": "SampleSchema1" 
}'

Response

Response Body

CreateSchemaCreatedBody

success response

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "response": {
    "message": "create schema",
    "message_code": "success"
  }
}

Delete a schema

DELETE /catalogs/{catalog_id}/schemas/{schema_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.schema.delete

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

catalog_id
Required*
string
Catalog name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+
schema_id
Required*
string
URL encoded Schema name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression .*

Query Parameters

engine_id
Required*
string
Engine name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'DELETE' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}?engine_id={engine_id}' -H 'accept: */*' -H 'AuthInstanceId: {instance_id}'

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

List all tables in a schema in a catalog for a given engine

GET /catalogs/{catalog_id}/schemas/{schema_id}/tables

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.table.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

catalog_id
Required*
string
catalog id

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+
schema_id
Required*
string
URL encoded schema name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+

Query Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'GET' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

Response

Response Body

TableCollection

tables list

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "tables": [
    "customer_data",
    "user_data"
  ]
}

Get details of a given table in a catalog and schema

GET /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.table.get

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

catalog_id
Required*
string
catalog id

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+
schema_id
Required*
string
URL encoded schema name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression .*
table_id
Required*
string
URL encoded table name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression .*

Query Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'GET' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

Response

Response Body

Table

GetColumns OK

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "columns": [
    {
      "column_name": "expenses",
      "comment": "expenses column",
      "extra": "varchar",
      "type": "varchar"
    }
  ],
  "table_name": "newtable,"
}

Delete table for a given schema and catalog

DELETE /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.table.delete

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

catalog_id
Required*
string
catalog id

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+
schema_id
Required*
string
URL encoded schema name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression .*
table_id
Required*
string
URL encoded table name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression .*

Query Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'DELETE' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}?engine_id={engine_id}' -H 'accept: */*' -H 'AuthInstanceId: {instance_id}'

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Rename table

PATCH /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.table.update

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

catalog_id
Required*
string
catalog id

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+
schema_id
Required*
string
URL encoded schema name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression .*
table_id
Required*
string
URL encoded table name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression .*

Query Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

application/json-patch+jsonTablePatch

Request body

curl example

curl -X 'PATCH' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}' -H 'Content-Type: application/json-patch+json' -d '{ 
  "new_table_name": "updated_table_name", 
}'

Response

Response Body

Table

GetColumns OK

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "columns": [
    {
      "column_name": "expenses",
      "comment": "expenses column",
      "extra": "varchar",
      "type": "varchar"
    }
  ],
  "table_name": "newtable,"
}

List all columns of a table in a given a schema for a given catalog

GET /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/columns

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.column.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

catalog_id
Required*
string
catalog id

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+
schema_id
Required*
string
URL encoded schema name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+
table_id
Required*
string
URL encoded schema name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+

Query Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'GET' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/columns?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

Response

Response Body

ColumnCollection

list of columns in a table

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "columns": [
    {
      "column_name": "customer_data"
    },
    {
      "column_name": "user_data"
    }
  ]
}

Add one or multiple columns to a table in a schema for a given catalog

POST /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/columns

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.column.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

catalog_id
Required*
string
catalog id

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+
schema_id
Required*
string
URL encoded schema name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+
table_id
Required*
string
URL encoded schema name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+

Query Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

ColumnPrototype

Request body

Examples:

View

curl example

curl -X 'POST' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/columns?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}' -H 'Content-Type: application/json' -d '{
  "bucket_name": "sample-bucket", 
  "custom_path": "sample-path", 
  "schema_name": "SampleSchema1" 
}'

Response

Response Body

ColumnCollection

list of columns in a table

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "columns": [
    {
      "column_name": "customer_data"
    },
    {
      "column_name": "user_data"
    }
  ]
}

Delete column in a table for a given schema and catalog

DELETE /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/columns/{column_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.column.delete

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

catalog_id
Required*
string
catalog id

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+
schema_id
Required*
string
URL encoded schema name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+
table_id
Required*
string
URL encoded schema name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+
column_id
Required*
string
URL encoded schema name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+

Query Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'DELETE' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}?engine_id={engine_id}' -H 'accept: */*' -H 'AuthInstanceId: {instance_id}'

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Update the given column - rename column

PATCH /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/columns/{column_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.column.update

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

catalog_id
Required*
string
catalog id

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+
schema_id
Required*
string
URL encoded schema name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+
table_id
Required*
string
URL encoded schema name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+
column_id
Required*
string
URL encoded schema name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+

Query Parameters

engine_id
Required*
string
engine id

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

application/json-patch+jsonColumnPatch

Request body

Examples:

View

curl example

curl -X 'PATCH' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}' -H 'Content-Type: application/json-patch+json' -d '{ 
  "add_columns": [ 
    { 
      "column_comment": "income column", 
      "column_name": "income", 
      "data_type": "varchar" 
    } 
  ], 
  "drop_columns": [ 
    { 
      "column_name": "expenditure" 
    } 
  ], 
  "new_table_name": "updated_table_name", 
  "rename_columns": [ 
    { 
      "column_name": "expenditure", 
      "new_column_name": "expenses" 
    } 
  ] 
}'

Response

Response Body

Column

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "column_name": "expenses,",
  "comment": "expenses column",
  "extra": "varchar,",
  "type": "varchar,"
}

List all table snapshots

GET /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/snapshots

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.table_snapshot.list

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

catalog_id
Required*
string
Catalog ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+
schema_id
Required*
string
Schema ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression .*
table_id
Required*
string
Table ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression .*

Query Parameters

engine_id
Required*
string
Engine name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'GET' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/snapshots?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

Response

Response Body

TableSnapshotCollection

TableSnapshot OK

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "snapshots": [
    {
      "committed_at": "1609379392",
      "operation": "alter",
      "snapshot_id": "2332342122211222",
      "summary": ""
    }
  ]
}

Rollback to a table snapshot

PUT /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/snapshots/{snapshot_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.table_rollback.set

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

catalog_id
Required*
string
Catalog ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+
schema_id
Required*
string
Schema ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression .*
table_id
Required*
string
Table ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression .*
snapshot_id
Required*
string
Snapshot ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+

Query Parameters

engine_id
Required*
string
Engine name

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'PUT' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/snapshots/{snapshot_id}?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

Response

Response Body

ReplaceSnapshotCreatedBody

success response

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "response": {
    "message": "Rollback to a table snapshot",
    "message_code": "success"
  }
}

Synchronize the external Iceberg table registration for a catalog identified by catalog_id.

PATCH /catalogs/{catalog_id}/sync

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.catalog_sync.update

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

catalog_id
Required*
string
catalog ID

Possible values: 1 ≤ length ≤ 1000, Value must match regular expression ^[a-zA-Z0-9\-_]+

Request Body

Required*

application/json-patch+jsonSyncCatalogs

Request body

curl example

curl -X 'PATCH' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/sync' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}' -H 'Content-Type: application/json-patch+json' -d '{ 
  "auto_add_new_tables": true, 
  "sync_iceberg_md": true 
}'

Response

Response Body

UpdateSyncCatalogOKBody

success response

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "response": {
    "message": "sync catalog",
    "message_code": "success"
  }
}

Get list milvus services

GET /milvus_services

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.milvus_service.read

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

curl example

curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance id}" https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/milvus_services"

Response

Response Body

MilvusServiceCollection

List milvus services

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "milvus_services": [
    {
      "actions": [
        "update",
        "delete"
      ],
      "created_by": "username@domain.com",
      "created_on": 1700201877,
      "grpc_port": 31501,
      "host_name": "<formation_id>.<kubernetes_cluster>.databases.appdomain.cloud",
      "https_port": 31012,
      "service_display_name": "test-milvus",
      "service_id": "milvus76",
      "status": "running",
      "status_code": 11,
      "type": "milvus"
    }
  ]
}

Create milvus service

POST /milvus_services

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.milvus_service.create

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*

MilvusServicePrototype

Create milvus service

curl example

curl -X 'POST' \'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/milvus_services' \-H 'accept: application/json' \-H "AuthInstanceId: {instance id}" \-H 'Content-Type: application/json' \-d '{  "bucket_name": "Sample bucket name",  "bucket_type": "Sample bucket type",  "description": "milvus service for running sql queries",  "origin": "native",  "root_path": "Sample path",  "service_display_name": "sampleService",  "tags": [    "tag1",    "tag2"  ],  "tshirt_size": "small",  "type": "milvus",   }'

Response

Response Body

MilvusService

milvus service details

Status Code

201
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 201

{
  "actions": [
    "update",
    "delete"
  ],
  "created_by": "username@domain.com",
  "created_on": 1700201877,
  "grpc_port": 31501,
  "host_name": "<formation_id>.<kubernetes_cluster>.databases.appdomain.cloud",
  "https_port": 31012,
  "milvus_service": null,
  "service_display_name": "test-milvus",
  "service_id": "milvus76",
  "status": "running",
  "status_code": 11,
  "type": "milvus"
}

Get milvus service

GET /milvus_services/{service_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.milvus_service.get

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

service_id
Required*
string
service id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'GET' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/milvus_services/{service_id}' -H 'accept: application/json' -H "AuthInstanceId: {instance id}" 

Response

Response Body

MilvusService

milvus service details

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "actions": [
    "update",
    "delete"
  ],
  "created_by": "username@domain.com",
  "created_on": 1700201877,
  "grpc_port": 31501,
  "host_name": "<formation_id>.<kubernetes_cluster>.databases.appdomain.cloud",
  "https_port": 31012,
  "milvus_service": null,
  "service_display_name": "test-milvus",
  "service_id": "milvus76",
  "status": "running",
  "status_code": 11,
  "type": "milvus"
}

Delete milvus service

DELETE /milvus_services/{service_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.milvus_service.delete

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

service_id
Required*
string
service id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

curl example

curl -X 'DELETE' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/milvus_services/{service_id}' -H 'accept: application/json' -H "AuthInstanceId: {instance id}" 

Response

Status Code

204
No Content
400
Error
401
Unauthorized
404
Not found
500
Internal server error

No Sample Response

This method does not specify any sample responses.

Update details of milvus service

PATCH /milvus_services/{service_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

lakehouse.milvus_service.update

Request

Custom Headers

AuthInstanceId
string
Instance ID

Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

service_id
Required*
string
service id

Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+

Request Body

Required*

application/json-patch+jsonMilvusServicePatch

Update milvus service

curl example

curl -X 'PATCH' 'https://{region}.lakehouse.cloud.ibm.com/lakehouse/api/v2/milvus_services/{service_id}' -H 'accept: application/json' -H 'Content-Type: application/json-patch+json' -H "AuthInstanceId: {instance id}" -d '{  "description": "updated description for milvus service",  "engine_display_name": "sampleService",  "tags": [    "tag1",    "tag2"  ]}'

Response

Response Body

MilvusService

milvus service details

Status Code

200
Success
400
Error
401
Unauthorized
404
Not found
500
Internal server error

Example responses

Status 200

{
  "actions": [
    "update",
    "delete"
  ],
  "created_by": "username@domain.com",
  "created_on": 1700201877,
  "grpc_port": 31501,
  "host_name": "<formation_id>.<kubernetes_cluster>.databases.appdomain.cloud",
  "https_port": 31012,
  "milvus_service": null,
  "service_display_name": "test-milvus",
  "service_id": "milvus76",
  "status": "running",
  "status_code": 11,
  "type": "milvus"
}

Introduction

Endpoint URLs

Authentication

Auditing

Error handling

Pagination

Methods

Get bucket registrations

Authorization

Request

Custom Headers

AuthInstanceId

Response

Response Body

bucket_registrations

Status Code

200

400

401

404

500

Register bucket

Authorization

Request

Custom Headers

AuthInstanceId

Request Body

bucket_details

bucket_type

description

managed_by

associated_catalog

bucket_display_name

region

tags

Response

Response Body

associated_catalog

bucket_type

created_by

created_on

description

managed_by

state

actions

bucket_details

bucket_display_name

bucket_id

region

tags

Status Code

201

400

401

404

500

Get bucket

Authorization

Request

Custom Headers

AuthInstanceId

Path Parameters

bucket_id

Response

Response Body

associated_catalog

bucket_type

created_by

created_on

description

managed_by

state

actions

bucket_details

bucket_display_name

bucket_id

region

tags

Status Code

200