IBM Cloud API Docs

Introduction

Last updated: 2023-07-10

IBM® watsonx.data is a data management solution for collecting, storing, querying, and analyzing all your enterprise data (structured, semi-structured, and unstructured) with a single unified data platform. It provides a flexible and reliable platform that is optimized to work on open data formats.

The api are in beta state and expected to change.

Endpoint URLs

The Watsonx.data API endpoint URL is based on your IBM Cloud Pak deployment URL. When you call the API, add the path for each method to form the complete API endpoint for your request. For example, if your instance is deployed at https://www.example.com:31843, you can access the APIs at https://www.example.com:31843/lakehouse/api/v2/{method}.

Authentication

Required Purpose: To work with the API, you must use an IBM Cloud Pak for Data API key (platform API key or instance API key), or IBM Cloud Pak for Data authorization token (platform token or instance token). The API key or token is used to determine the actions that a user or service ID has access to when they use the API.

You can generate an API key or authorization token for an authenticated user by using the Cloud Pak for Data APIs. For more information, see Generating an authorization token or API key.

To use the API, add a valid API key or token to the HTTP Authorization request header. For token, -H 'Authorization: Bearer {TOKEN}'. For API key, you need to firstly encode username together with API key: echo "<username>:<api_key>" | base64. Then, you can use the ZenApiKey header to authenticate to an endpoint: -H "Authorization: ZenApiKey <encoded value>".

To retrieve your access token:

For most Cloud Pak for Data endpoints, you can use the generated API key in the authorization header of subsequent API calls.

curl -H "Authorization: ZenApiKey <token>" "https://<cpd_instance_route>/<endpoint>"

If the ZenApiKey token is not supported, use a generated Bearer token in the authorization header of subsequent API calls.

curl -H "Authorization: Bearer <token>" "https://<cpd_instance_route>/<endpoint>"

Auditing

Required if applicable Purpose: Describes that the API generates auditing events that can be consumed by the Activity Tracker service, and links to your related product docs for more information. Required only if any of the API's methods is enabled to generate auditing events.

Example:

You can monitor API activity within your account by using the IBM Cloud Activity Tracker service. Whenever an API method is called, an event is generated that you can then track and audit from within Activity Tracker. The specific event type is listed for each individual method.

For more information about how to track Certificate Manager activity, see Auditing events for Certificate Manager.

Error handling

This API uses standard HTTP response codes to indicate whether a method completed successfully. A 200 response indicates success. A 400 type response indicates a failure, and a 500 type response indicates an internal system error.

HTTP Error Code Description Recovery
200 Success The request was successful.
201 Created The requested resource successfully created in a synchronous manner.
204 No Content The server successfully processed the request and is not returning any content.
400 Bad Request The input parameters in the request body are either incomplete or in the wrong format. Be sure to include all required parameters in your request.
401 Unauthorized You are not authorized to make this request. Log in to IBM Cloud and try again. If this error persists, contact the account owner to check your permissions.
403 Forbidden The supplied authentication is not authorized to access '{namespace}'.
404 Not Found The requested resource could not be found.
409 Conflict The entity is already in the requested state.
500 Internal Server Error Your request could not be processed. Wait a few minutes and try again.

Pagination

Some API requests might return many results. To avoid performance issues, these results are returned one page at a time, with a limited number of results on each page. GET requests for the following resources use pagination:

  • /v1/statement

For the request that uses pagination, the response does include following urls to make subsequent requests :

  • infoUri: The URL for requesting the first page of results.
  • nextUri: The URL for requesting the next page of results.

Methods

Get bucket registrations

Get list of registered buckets

GET /bucket_registrations

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.bucket_registration.list

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

  • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/bucket_registrations"

Response

Response Body

BucketRegistrationCollection

List bucket registrations

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "bucket_registrations": [
    {
      "actions": [
        "browse",
        "view",
        "modify",
        "create",
        "grant",
        "revoke",
        "update",
        "remove",
        "activate",
        "register"
      ],
      "associated_catalog": {
        "catalog_name": "iceberg_catalog",
        "catalog_tags": []
      },
      "bucket_details": {
        "bucket_name": "iceberg-bucket",
        "endpoint": "http://xyz-minio-svc:9000"
      },
      "bucket_display_name": "hive-bucket",
      "bucket_id": "iceberg-bucket",
      "bucket_type": "minio",
      "created_by": "user",
      "created_on": "1699457595",
      "description": "default bucket",
      "managed_by": "ibm",
      "region": "us-south",
      "state": "active",
      "tags": [
        "tag1",
        "tag2"
      ]
    },
    {
      "actions": [
        "browse",
        "view",
        "modify",
        "create",
        "grant",
        "revoke",
        "update",
        "remove",
        "activate",
        "register"
      ],
      "associated_catalog": {
        "catalog_name": "hive_data",
        "catalog_tags": []
      },
      "bucket_details": {
        "bucket_name": "hive-bucket",
        "endpoint": "http://xyz-minio-svc:9000"
      },
      "bucket_display_name": "hive-bucket",
      "bucket_id": "hive-bucket",
      "bucket_type": "minio",
      "created_by": "user",
      "created_on": "1699457595",
      "description": "default bucket",
      "managed_by": "ibm",
      "region": "us-south",
      "state": "active",
      "tags": [
        "tag1",
        "tag2"
      ]
    }
  ]
}

Register bucket

Register a new bucket

POST /bucket_registrations

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.bucket_registration.create

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*
BucketRegistrationPrototype

Request body

  • curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json" -d '{ "bucket_details": { "access_key": "<access_key>", "bucket_name": "sample-bucket", "endpoint": "https://s3.{region}.cloud-object-storage.appdomain.cloud/", "secret_key": "secret_key" }, "bucket_display_name": "sample-bucket-displayname", "bucket_tags": [ "read customer data", "write customer data'" ], "bucket_type": "ibm_cos", "catalog_name": "sampleCatalog", "catalog_tags": [ "catalog_tag_1", "catalog_tag_2" ], "description": "COS bucket for customer data", "managed_by": "ibm", "region": "us-south", "state": "active", "table_type": "iceberg" }' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/bucket_registrations"

Response

Response Body

BucketRegistration

Bucket

Status Code

  • 201

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "create",
    "update"
  ],
  "associated_catalog": {
    "catalog_name": "hive_data",
    "catalog_type": "hive"
  },
  "bucket_display_name": "samplebucketdisplayname",
  "bucket_id": "samplebucketid",
  "bucket_name": "samplebucket",
  "bucket_type": "minio",
  "created_by": "username@domain.com",
  "created_on": "1699457595",
  "description": "default bucket",
  "endpoint": "https://s3.<region>.cloud-object-storage.appdomain.cloud/",
  "managed_by": "ibm",
  "region": "us-south",
  "state": "active",
  "tags": [
    "tag1",
    "tag2"
  ]
}

Get bucket

Get a registered bucket

GET /bucket_registrations/{bucket_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.bucket_registration.get

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • bucket_id
    Required*
    string

    bucket id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/bucket_registrations/{bucket_id}"

Response

Response Body

BucketRegistration

Bucket

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "create",
    "update"
  ],
  "associated_catalog": {
    "catalog_name": "hive_data",
    "catalog_type": "hive"
  },
  "bucket_display_name": "samplebucketdisplayname",
  "bucket_id": "samplebucketid",
  "bucket_name": "samplebucket",
  "bucket_type": "minio",
  "created_by": "username@domain.com",
  "created_on": "1699457595",
  "description": "default bucket",
  "endpoint": "https://s3.<region>.cloud-object-storage.appdomain.cloud/",
  "managed_by": "ibm",
  "region": "us-south",
  "state": "active",
  "tags": [
    "tag1",
    "tag2"
  ]
}

Deregister Bucket

Deregister a bucket

DELETE /bucket_registrations/{bucket_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.bucket_registration.delete

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • bucket_id
    Required*
    string

    bucket id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X DELETE -H "accept:  */*" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/bucket_registrations/{bucket_id}"

Response

Status Code

  • 204

    No Content

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

No Sample Response

This method does not specify any sample responses.

Update bucket

Update bucket details & credentials

PATCH /bucket_registrations/{bucket_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.bucket_registration.update

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • bucket_id
    Required*
    string

    bucket id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Request Body

Required*
application/merge-patch+jsonBucketRegistrationPatch

Request body

Examples: 
{
  "bucket_display_name": "samplebucketdisplayname",
  "bucket_id": "samplebucketid",
  "created_by": "username@domain.com",
  "description": "default bucket",
  "tags": [
    "tag1",
    "tag2"
  ]
}
  • curl -X PATCH -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json" -d '{ "access_key": "<access_key>", "bucket_display_name": "sample-bucket-displayname", "description": "COS bucket for customer data", "secret_key": "<secret_key>", "tags": [ "testbucket", "userbucket" ] }' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/bucket_registrations/{bucket_id}"

Response

Response Body

BucketRegistration

Bucket

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "create",
    "update"
  ],
  "associated_catalog": {
    "catalog_name": "hive_data",
    "catalog_type": "hive"
  },
  "bucket_display_name": "samplebucketdisplayname",
  "bucket_id": "samplebucketid",
  "bucket_name": "samplebucket",
  "bucket_type": "minio",
  "created_by": "username@domain.com",
  "created_on": "1699457595",
  "description": "default bucket",
  "endpoint": "https://s3.<region>.cloud-object-storage.appdomain.cloud/",
  "managed_by": "ibm",
  "region": "us-south",
  "state": "active",
  "tags": [
    "tag1",
    "tag2"
  ]
}

Activate Bucket

Activate a registered bucket

POST /bucket_registrations/{bucket_id}/activate

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.bucket_registration_activate.add

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • bucket_id
    Required*
    string

    bucket id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -d "" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/bucket_registrations/{bucket_id}/activate"

Response

Response Body

CreateActivateBucketCreatedBody

Activate bucket

Status Code

  • 201

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "response": {
    "message": "Activate bucket",
    "message_code": "success"
  }
}

Add bucket catalog

Add bucket catalog

POST /bucket_registrations/{bucket_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.catalog.create

Request

Custom Headers

  • AuthInstanceId
    string

    watsonx.data instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • bucket_id
    Required*
    string

    bucket id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\\._\/-]+$

Request Body

Required*
BucketCatalog

Request body

Examples: 
{
  "catalog_name": "iceberg_catalog",
  "catalog_type": "iceberg"
}
  • curl -X POST-H "accept: application/json"-H "AuthInstanceId: {instance_id}"-H "Content-Type: application/json"-d '{  "catalog_name": "test_cqatalog5",  "catalog_tags": ["test"],  "catalog_type": "iceberg"}'"https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/bucket_registrations"

Response

Response Body

SuccessResponse

Response of success

Status Code

  • 201

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "message": "Successful message",
  "message_code": "success"
}

Deactivate Bucket

Deactivate a bucket

DELETE /bucket_registrations/{bucket_id}/deactivate

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.bucket_registration_activate.remove

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • bucket_id
    Required*
    string

    bucket id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X DELETE -H "accept: */*" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/bucket_registrations/{bucket_id}/deactivate"

Response

Status Code

  • 204

    No Content

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

No Sample Response

This method does not specify any sample responses.

List bucket objects

Fetch all objects from a given bucket

GET /bucket_registrations/{bucket_id}/objects

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.bucket_registration_object.list

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • bucket_id
    Required*
    string

    bucket id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/bucket_registrations/{bucket_id}/objects"

Response

Response Body

BucketRegistrationObjectCollection

List bucket objects

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "objects": [
    "/path/file1.json",
    "/path/file2.txt",
    "/path/file3.csv",
    "/path/file4.parquet"
  ]
}

Generate generate_benchmark_report specific to storage

Generate generate_benchmark_report specific to storage

GET /generate_benchmark_report

Request

Custom Headers

  • AuthInstanceId
    string

    watsonx.data instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Query Parameters

  • bucket_name
    Required*
    string

    bucket name

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+

  • pod_name
    Required*
    string

    specify the pod name of the respective presto pod like coordinator(single node), coordinator-blue-0,worker-0(multinode)

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9.\\-]+$

  • file_count
    string

    configure file_count and file_size for benchmarking

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+

  • file_size
    string

    configure file_count and file_size for benchmarking

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+

  • curl -X GET -H "content-type: application/json" -H "LhInstanceId: {instance_id}" "https://{cpd_url}/lakehouse/api/v2/1737546241011604/generate_benchmark_report?bucket_name=iceberg-bucket&engine_id=presto-01&pod_name=coordinator-blue-0"

Response

Response Body

GenerateBenchmarkReportOKBody

Generate benchmark report

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "response": {
    "message": "bucket benchmarking is in progress.Trigger /generate_benchmark_report/status?req_id=<xx> for the status",
    "req_id": "c3a3dbd4-98f8-4268-9128-12339ca8566b",
    "status": "success"
  }
}

Get generate_benchmark_report status

Generate generate_benchmark_report status

GET /generate_benchmark_report/status/{req_id}

Request

Custom Headers

  • AuthInstanceId
    string

    watsonx.data instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • req_id
    Required*
    string

    request_id

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+

Query Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+

  • bucket_name
    Required*
    string

    bucket name

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+

  • pod_name
    Required*
    string

    specify the pod name of the respective presto pod like coordinator(single node), coordinator-blue-0,worker-0(multinode)

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9.\\-]+$

  • curl -X GET -H "content-type: application/json" -H "LhInstanceId: {instance_id}" "https://{cpd_url}/lakehouse/api/v2/1737546241011604/generate_benchmark_report/status/{req_id}?bucket_name=iceberg-bucket&engine_id=presto-01&pod_name=coordinator-blue-0"

Response

Response Body

BenchmarkStatusResponse

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

No Sample Response

This method does not specify any sample responses.

Get databases

Get list of databases

GET /database_registrations

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.database_registration.list

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

  • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/database_registrations"

Response

Response Body

DatabaseRegistrationCollection

list database registrations

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "database_registrations": [
    {
      "actions": [
        "update",
        "delete"
      ],
      "associated_catalog": {
        "catalog_name": "hive_data",
        "catalog_tags": [
          "catalog_tag_1",
          "catalog_tag_2"
        ],
        "catalog_type": "hive"
      },
      "created_by": "user1@bim.com",
      "created_on": "1686792721",
      "database_details": {
        "database_name": "new_database",
        "hostname": "netezza://ps.fyre.com",
        "password": "samplepassword",
        "port": 4353,
        "ssl": true,
        "tables": "netezza_table_name",
        "username": "sampleuser"
      },
      "database_display_name": "new_database",
      "database_id": "new_database_id",
      "database_type": "netezza",
      "description": "Description of the external Database",
      "tags": [
        "testdatabase",
        "userdatabase"
      ]
    }
  ]
}

Add/Create database

Add or create a new database

POST /database_registrations

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.database_registration.create

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*
DatabaseRegistrationPrototype

Request body

  • curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json" -d '{
  "database_display_name": "testdb",
  "database_properties": [],
  "description": "",
  "tags": [],
  "database_type": "db2",
  "database_details": {
    "database_name": "testdb",
    "hostname": "9.30.252.96",
    "port": 50000,
    "username": "db2username",
    "password": "password",
    "ssl": false
  },
  "associated_catalog": {
    "catalog_name": "db2_catalog_testdb"
  }
}' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/database_registrations"

Response

Response Body

DatabaseRegistration

database registration object

Status Code

  • 201

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "update",
    "delete"
  ],
  "associated_catalog": {
    "catalog_name": "iceberg_data",
    "catalog_type": "iceberg"
  },
  "created_by": "user1@bim.com",
  "created_on": "1686792721",
  "database_details": {
    "database_name": "new_database",
    "hostname": "netezza://abc.efg.com",
    "password": "samplepassword",
    "port": 4353,
    "ssl": true,
    "tables": "netezza_table_name",
    "username": "sampleuser"
  },
  "database_display_name": "new_database",
  "database_id": "new_database_id",
  "database_type": "netezza",
  "description": "Description of the database",
  "tags": [
    "testdatabase",
    "userdatabase"
  ]
}

Get database

Get a registered databases

GET /database_registrations/{database_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.database_registration.get

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • database_id
    Required*
    string

    database id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/database_registrations/{database_id}"

Response

Response Body

DatabaseRegistration

database registration object

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "update",
    "delete"
  ],
  "associated_catalog": {
    "catalog_name": "iceberg_data",
    "catalog_type": "iceberg"
  },
  "created_by": "user1@bim.com",
  "created_on": "1686792721",
  "database_details": {
    "database_name": "new_database",
    "hostname": "netezza://abc.efg.com",
    "password": "samplepassword",
    "port": 4353,
    "ssl": true,
    "tables": "netezza_table_name",
    "username": "sampleuser"
  },
  "database_display_name": "new_database",
  "database_id": "new_database_id",
  "database_type": "netezza",
  "description": "Description of the database",
  "tags": [
    "testdatabase",
    "userdatabase"
  ]
}

Delete database

Delete a database

DELETE /database_registrations/{database_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.database_registration.delete

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • database_id
    Required*
    string

    database id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X DELETE -H "accept: */*" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/database_registrations/{database_id}"

Response

Status Code

  • 204

    No Content

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

No Sample Response

This method does not specify any sample responses.

Update database

Update database details

PATCH /database_registrations/{database_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.database_registration.update

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • database_id
    Required*
    string

    database id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Request Body

Required*
application/merge-patch+jsonDatabaseRegistrationPatch

Request body

  • curl -X PATCH -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/merge-patch+json" -d '{ "database_details": { "password": "samplepassword", "username": "sampleuser" }, "database_display_name": "new_database", "description": "External database description", "tags": [ "testdatabase", "userdatabase" ] }' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/database_registrations/{database_id}"

Response

Response Body

DatabaseRegistration

database registration object

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "update",
    "delete"
  ],
  "associated_catalog": {
    "catalog_name": "iceberg_data",
    "catalog_type": "iceberg"
  },
  "created_by": "user1@bim.com",
  "created_on": "1686792721",
  "database_details": {
    "database_name": "new_database",
    "hostname": "netezza://abc.efg.com",
    "password": "samplepassword",
    "port": 4353,
    "ssl": true,
    "tables": "netezza_table_name",
    "username": "sampleuser"
  },
  "database_display_name": "new_database",
  "database_id": "new_database_id",
  "database_type": "netezza",
  "description": "Description of the database",
  "tags": [
    "testdatabase",
    "userdatabase"
  ]
}

List other engines

list all other engine details

GET /other_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.other_engine.list

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

  • curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/other_engines"

Response

Response Body

OtherEngineCollection

list other engines

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "other_engines": [
    {
      "created_by": "<username>@<domain>.com",
      "created_on": 163788384993,
      "description": "other engine for running queries",
      "engine_details": {
        "connection_string": "https://other-connection-string-sample.com",
        "engine_type": "netezza"
      },
      "engine_display_name": "sampleEngine",
      "engine_id": "sampleEngine123",
      "origin": "external",
      "status": "registered",
      "type": "other"
    }
  ]
}

Create other engine

Create a new engine

POST /other_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.other_engine.create

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*
OtherEnginePrototype

Create other Engine Body

  • curl -X POST -H "content-type: application/json" -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -d '{ "description": "external engine description", "engine_details": { "connection_string": "1.2.3.4", "engine_type": "netezza", "metastore_host": "1.2.3.4" }, "engine_display_name": "sampleEngine01", "tags": [ "tag1", "tag2" ] }' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/other_engines"

Response

Response Body

OtherEngine

external engine details

Status Code

  • 201

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "OtherEngine": {
    "created_by": "<username>@<domain>.com",
    "created_on": 163788384993,
    "description": "other engine for running sql queries",
    "engine_display_name": "sampleEngine",
    "engine_id": "sampleEngine123",
    "origin": "external",
    "status": "registered",
    "type": "other"
  }
}

Delete engine

Delete an engine from lakehouse

DELETE /other_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.other_engine.delete

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X DELETE -H "accept: */*" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/other_engines/{engine_id}"

Response

Status Code

  • 204

    No Content

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

No Sample Response

This method does not specify any sample responses.

Get all existing Integrations

Get all existing Integrations

GET /integrations

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.integration.listall

Request

Custom Headers

  • AuthInstanceId
    string

    watsonx.data instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

  • Secret
    string

    API Authentication service token

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

Query Parameters

  • service_type
    string

    service_type

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^.*$

  • state
    string[]

    state

    Possible values: 0 ≤ number of items ≤ 10000, 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {token}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/integrations"

Response

Response Body

IntegrationCollection

list all existing integrations

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "integrations": [
    {
      "enable_data_policy_within_wxd": true,
      "integration_id": "ranger952",
      "modified_at": 1234567890,
      "modified_by": "admin",
      "password": "pasword_encrypted",
      "resource": "resource_name",
      "service_type": "ranger",
      "state": "active",
      "url": "http://abcd.efgh.com:9876/",
      "username": "username"
    },
    {
      "apikey": "apikey_encrypted",
      "config_properties": "ikc-env.password=apikey_encrypted\nikc-env.url=https://abcd.efgh.com\nikc-username=test\nikc-enabled-catalogs=storage_catalog_1,storage_catalog_2\nlh-unique-identifier=1234567890123456\nlh-crn=0123456789012345",
      "governance_properties": "query-governance.name=external",
      "integration_id": "ikc806",
      "modified_at": 1234567890,
      "modified_by": "admin",
      "service_type": "ikc",
      "state": "active",
      "storage_catalogs": [
        "storage_catalog_1",
        "storage_catalog_2"
      ],
      "url": "https://abcd.efgh.com",
      "username": "username"
    }
  ]
}

To register an integration

To register an integration

POST /integrations

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.integration.create

Request

Custom Headers

  • AuthInstanceId
    string

    watsonx.data instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*
IntegrationPrototype

Integration registration parameters

Examples: 
{
  "url": "http://abcd.efgh.com:9876/",
  "service_type": "ranger",
  "username": "username",
  "password": "password",
  "resource": "resource_name",
  "enable_data_policy_within_wxd": false
}
{
  "url": "https://abcd.efgh.com",
  "service_type": "ikc",
  "apikey": "apikey",
  "storage_catalogs": [
    "storage_catalog_1",
    "storage_catalog_2"
  ],
  "username": "username"
}
{
  "url": "https://abcd.efgh.com",
  "service_type": "databand",
  "access_token": "accessToken"
}
{
  "apikey": "apikey",
  "username": "username",
  "service_type": "manta",
  "url": "https://abcd.efgh.com"
}
  • curl -X POST -H "accept: application/json" -H "AuthInstanceId: {token}" -H "Content-Type: application/json" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/integrations" -d '{  "url": "http://lh-ranger-test1.fyre.ibm.com:6080",  "service_type": "ranger",  "username": "admin",  "password": "Rangeradmin1",  "resource": "default_policy_donot_change",  "enable_data_policy_within_wxd": false}'
  • curl -X POST \-H "accept: application/json" \ 
-H "AuthInstanceId: {token}" \ 
-H "Content-Type: application/json" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/integrations" \ 
-d '{  "url": "https://cpd-wkc.apps.wkcwxd.cp.fyre.ibm.com",  "service_type": "ikc",  "apikey": "YWRtaW46VmpSb3JOV2ZvRFE1TGdLWUpiNGd5YmRNaW1Gc25CN0tjdGtlSmZXbwo=",  "storage_catalogs": "iceberg_data,wxd_system_data",  "username": "test"}'

Response

Response Body

Integration

Integration

Status Code

  • 201

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "integration_id": "ranger952",
  "modified_by": "admin",
  "password": "password_encrypted",
  "resource": "resource_name",
  "service_type": "ranger",
  "state": "active",
  "url": "http://abcd.efgh.com:9876/",
  "username": "username"
}
  • {
  "apikey": "apikey_encrypted",
  "integration_id": "ikc806",
  "modified_by": "admin",
  "service_type": "ikc",
  "state": "active",
  "storage_catalogs": [
    "storage_catalog_1",
    "storage_catalog_2"
  ],
  "url": "https://abcd.efgh.com",
  "username": "username"
}
  • {
  "access_token": "accessToken",
  "integration_id": "databand123",
  "modified_by": "username@domain.com",
  "service_type": "databand",
  "url": "https://abcd.efgh.com"
}
  • {
  "apikey": "apikey",
  "username": "username",
  "integration_id": "manta123",
  "modified_by": "username@domain.com",
  "service_type": "manta",
  "url": "https://abc.def.ibm.com"
}

Get an Integration

Get an Integration

GET /integrations/{integration_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.integration.get

Request

Custom Headers

  • AuthInstanceId
    string

    watsonx.data instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

  • Secret
    string

    API Authentication service token

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

Path Parameters

  • integration_id
    Required*
    string

    integration_id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^.*$

  • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {token}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/integrations/{service_type}"

Response

Response Body

Integration

Integration

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "enable_data_policy_within_wxd": true,
  "integration_id": "ranger952",
  "modified_at": 1234567890,
  "modified_by": "admin",
  "password": "pasword_encrypted",
  "resource": "resource_name",
  "service_type": "ranger",
  "state": "active",
  "url": "http://abcd.efgh.com:9876/",
  "username": "username"
}
  • {
  "apikey": "apikey_encrypted",
  "config_properties": "ikc-env.password=apikey_encrypted\nikc-env.url=https://abcd.efgh.com\nikc-username=test\nikc-enabled-catalogs=storage_catalog_1,storage_catalog_2\nlh-unique-identifier=1234567890123456\nlh-crn=0123456789012345",
  "governance_properties": "query-governance.name=external",
  "integration_id": "ikc806",
  "modified_at": 1234567890,
  "modified_by": "admin",
  "service_type": "ikc",
  "state": "active",
  "storage_catalogs": [
    "storage_catalog_1",
    "storage_catalog_2"
  ],
  "url": "https://abcd.efgh.com",
  "username": "username"
}
  • {
  "access_token": "accessToken",
  "integration_id": "databand123",
  "modified_at": 1234567890,
  "modified_by": "username@domain.com",
  "service_type": "databand",
  "state": "inactive",
  "url": "https://abcd.efgh.com"
}
  • {
  "apikey": "apikey",
  "integration_id": "manta123",
  "manta_url": "https://api.abc.def.ibm.com/gov_lineage/v2/lineage_events/openlineage",
  "modified_at": 1234567890,
  "modified_by": "username@domain.com",
  "service_type": "manta",
  "state": "inactive",
  "url": "https://abc.def.ibm.com",
  "username": "username",
  "zen_apikey": "zenApikeyEncrypted"
}

Remove an Integration

Remove an Integration

DELETE /integrations/{integration_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.integration.delete

Request

Custom Headers

  • AuthInstanceId
    string

    watsonx.data instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • integration_id
    Required*
    string

    integration_id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^.*$

  • curl -X DELETE -H "accept: application/json" -H "AuthInstanceId: {token}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/integrations/{service_type}"

Response

Status Code

  • 204

    No Content

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

No Sample Response

This method does not specify any sample responses.

Update an existing Integration

Update an existing Integration

PATCH /integrations/{integration_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.integration.update

Request

Custom Headers

  • AuthInstanceId
    string

    watsonx.data instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • integration_id
    Required*
    string

    integration_id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^.*$

Request Body

Required*
application/merge-patch+jsonIntegrationPatch

Integration update parameters

Examples: 
{
  "url": "http://abcd.efgh.com:9876/",
  "username": "username",
  "resource": "resource_name",
  "enable_data_policy_within_wxd": true
}
{
  "url": "https://abcd.efgh.com",
  "apikey": "apikey",
  "storage_catalogs": [
    "storage_catalog_1"
  ],
  "username": "username"
}
{
  "url": "https://abcd.efgh.com",
  "access_token": "accessToken"
}
{
  "username": "username",
  "url": "https://abcd.efgh.com",
  "apikey": "apikey"
}
  • curl -X PATCH -H "accept: application/json" -H "AuthInstanceId: {token}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/integrations/{integration_id}" -d '{  "url": "http://lh-ranger-test1.fyre.ibm.com:6080/",  "username": "admin2"  "resource": "default_policy_donot_change",  "enable_data_policy_within_wxd": true,}'
  • curl -X PATCH \-H "accept: application/json" -H "AuthInstanceId: {token}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/integrations/{integration_id}" -d '{  "url": "https://cpd-wkc.apps.wkcwxd.cp.fyre.ibm.com",  "apikey": "YWRtaW46VmpSb3JOV2ZvRFE1TGdLWUpiNGd5YmRNaW1Gc25CN0tjdGtlSmZXbwo=",  "storage_catalogs": "iceberg_data",  "username": "test"}'

Response

Response Body

Integration

Integration

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "enable_data_policy_within_wxd": true,
  "integration_id": "ranger952",
  "modified_at": 1234567890,
  "modified_by": "admin",
  "password": "pasword_encrypted",
  "resource": "resource_name",
  "service_type": "ranger",
  "state": "active",
  "url": "http://abcd.efgh.com:9876/",
  "username": "username"
}
  • {
  "apikey": "apikey_encrypted",
  "config_properties": "ikc-env.password=apikey_encrypted\nikc-env.url=https://abcd.efgh.com\nikc-username=test\nikc-enabled-catalogs=storage_catalog_1,storage_catalog_2\nlh-unique-identifier=1234567890123456\nlh-crn=0123456789012345",
  "governance_properties": "query-governance.name=external",
  "integration_id": "ikc806",
  "modified_at": 1234567890,
  "modified_by": "admin",
  "service_type": "ikc",
  "state": "active",
  "storage_catalogs": [
    "storage_catalog_1"
  ],
  "url": "https://abcd.efgh.com",
  "username": "username"
}
  • {
  "access_token": "accessToken",
  "integration_id": "databand123",
  "modified_at": 1234567890,
  "modified_by": "username@domain.com",
  "service_type": "databand",
  "state": "inactive",
  "url": "https://abcd.efgh.com"
}
  • {
  "apikey": "apikey",
  "integration_id": "manta123",
  "manta_url": "https://api.abc.def.ibm.com/gov_lineage/v2/lineage_events/openlineage",
  "modified_at": 1234567890,
  "modified_by": "username@domain.com",
  "service_type": "manta",
  "state": "inactive",
  "url": "https://abc.def.ibm.com",
  "username": "username",
  "zen_apikey": "zenApikeyEncrypted"
}

Get list of db2 engines

Get list of all db2 engines

GET /db2_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.db2_engine.list

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

  • curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/db2_engines"

Response

Response Body

Db2EngineCollection

list db2 engines

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "db2_engines": [
    {
      "created_by": "user@test.com",
      "created_on": 1700322436,
      "description": "db2 engine for running sql queries",
      "engine_details": {
        "connection_string": "jdbc:db2://<hostname>:<port>/<database>",
        "metastore_host": "thrift://mh-connection-string-sample.com"
      },
      "engine_display_name": "db2",
      "engine_id": "db2505",
      "origin": "external",
      "status": "REGISTERED",
      "tags": [
        "tag1",
        "tag2"
      ],
      "type": "db2"
    }
  ]
}

Create db2 engine

Create a new db2 engine

POST /db2_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.db2_engine.create

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*
Db2EnginePrototype

Create db2 engine body

  • curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json" -d '{ "description": "db2 engine description", "engine_details": { "connection_string": "1.2.3.4" }, "engine_display_name": "sampleEngine", "origin": "external", "tags": [ "tag1", "tag2" ] }' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/db2_engines"

Response

Response Body

Db2Engine

Db2 engine details

Status Code

  • 201

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "Db2Engine": null,
  "created_by": "user@test.com",
  "created_on": 1700322469,
  "description": "updated description for db2 engine.",
  "engine_details": {
    "connection_string\"": "jdbc:db2://localhost:5480/database",
    "metastore_host": "thrift://mh-connection-string-sample.com"
  },
  "engine_display_name": "sample db2 Engine Display Name",
  "engine_id": "sample db2 Engine Name",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "db2"
}

Delete db2 engine

Delete a db2 engine

DELETE /db2_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.db2_engine.delete

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X DELETE -H "accept: */*" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/db2_engines/{engine_id}"

Response

Status Code

  • 204

    No Content

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

No Sample Response

This method does not specify any sample responses.

Update db2 engine

Update details of db2 engine

PATCH /db2_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.db2_engine.update

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Request Body

Required*
application/merge-patch+jsonDb2EnginePatch

Update Engine Body

  • curl -X PATCH -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/merge-patch+json" -d '{ "description": "db2 engine updated description", "engine_display_name": "sampleEngine", "tags": [ "tag1", "tag2" ] }' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/db2_engines/{engine_id}"

Response

Response Body

Db2Engine

Db2 engine details

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "Db2Engine": null,
  "created_by": "user@test.com",
  "created_on": 1700322469,
  "description": "updated description for db2 engine.",
  "engine_details": {
    "connection_string\"": "jdbc:db2://localhost:5480/database",
    "metastore_host": "thrift://mh-connection-string-sample.com"
  },
  "engine_display_name": "sample db2 Engine Display Name",
  "engine_id": "sample db2 Engine Name",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "db2"
}

Get list of netezza engines

Get list of all netezza engines

GET /netezza_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.netezza_engine.list

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

  • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/netezza_engines"

Response

Response Body

NetezzaEngineCollection

list Netezza engines

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "netezza_engines": [
    {
      "created_by": "user@test.com",
      "created_on": 1700322469,
      "description": "netezza engine for running sql queries",
      "engine_details": {
        "connection_string": "jdbc:netezza://localhost:5480/database",
        "metastore_host": "thrift://mh-connection-string-sample.com"
      },
      "engine_display_name": "netezza",
      "engine_id": "netezza170",
      "origin": "external",
      "status": "REGISTERED",
      "tags": [
        "tag1",
        "tag2"
      ],
      "type": "netezza"
    }
  ]
}

Create netezza engine

Create a new netezza engine

POST /netezza_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.netezza_engine.create

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*
NetezzaEnginePrototype

Create netezza engine body

  • curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json" -d '{ "description": "netezza engine description", "engine_details": { "connection_string": "1.2.3.4" }, "engine_display_name": "sampleEngine", "origin": "external", "tags": [ "tag1", "tag2" ]}' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/netezza_engines"

Response

Response Body

NetezzaEngine

Netezza engine details

Status Code

  • 201

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "NetezzaEngine": null,
  "created_by": "user@test.com",
  "created_on": 1700322469,
  "description": "updated description for netezza engine.",
  "engine_details": {
    "connection_string\"": "jdbc:netezza://localhost:5480/database",
    "metastore_host": "thrift://mh-connection-string-sample.com"
  },
  "engine_display_name": "sample Netezza Engine Display Name",
  "engine_id": "sample Netezza Engine Name",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "netezza"
}

Delete netezza engine

Delete a netezza engine

DELETE /netezza_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.netezza_engine.delete

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X DELETE -H "accept: */* " -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/netezza_engines/{engine_id}"

Response

Status Code

  • 204

    No Content

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

No Sample Response

This method does not specify any sample responses.

Update netezza engine

Update details of netezza engine

PATCH /netezza_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.netezza_engine.update

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Request Body

Required*
application/merge-patch+jsonNetezzaEnginePatch

Update Engine Body

  • curl -X PATCH -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/merge-patch+json" -d '{ "description": "netezza engine updated description", "engine_display_name": "sampleEngine", "tags": [ "tag1", "tag2" ] }' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/netezza_engines/{engine_id}"

Response

Response Body

NetezzaEngine

Netezza engine details

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "NetezzaEngine": null,
  "created_by": "user@test.com",
  "created_on": 1700322469,
  "description": "updated description for netezza engine.",
  "engine_details": {
    "connection_string\"": "jdbc:netezza://localhost:5480/database",
    "metastore_host": "thrift://mh-connection-string-sample.com"
  },
  "engine_display_name": "sample Netezza Engine Display Name",
  "engine_id": "sample Netezza Engine Name",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "netezza"
}

Get list of prestissimo engines

Get list of all prestissimo engines

GET /prestissimo_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.prestissimo_engine.list

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Response

Response Body

PrestissimoEngineCollection

list Prestissimo Engines

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "prestissimo_engines": [
    {
      "actions": [
        "view",
        "use",
        "update",
        "select",
        "access_ui",
        "associate",
        "disassociate",
        "restart",
        "pause",
        "resume",
        "grant",
        "revoke",
        "delete",
        "create",
        "scale"
      ],
      "associated_catalogs": [
        "hive_data"
      ],
      "build_version": "1.1.0.0.0",
      "coordinator": {
        "node_type": "bx2.4x16",
        "quantity": 1
      },
      "engine_display_name": "starter",
      "engine_id": "prestissimo511",
      "external_host_name": "your-hostname.apps.your-domain.com",
      "group_id": "prestissimo511",
      "host_name": "1234-xyz456-abc4321.databases.appdomain.cloud",
      "port": 30156,
      "size_config": "starter",
      "status": "running",
      "status_code": 0,
      "tags": [
        "tag1",
        "tag2"
      ],
      "type": "prestissimo",
      "version": "v0.282",
      "worker": {
        "node_type": "bx2.4x16",
        "quantity": 1
      }
    }
  ]
}

Create prestissimo engine

Create a new prestissimo engine

POST /prestissimo_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.prestissimo_engine.create

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*
PrestissimoEnginePrototype

Create Engine Body

Response

Response Body

PrestissimoEngine

EngineDetail

Status Code

  • 201

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "update",
    "delete"
  ],
  "associated_catalogs": [
    "hive_data"
  ],
  "coordinator": {
    "node_type": "worker",
    "quantity": 1
  },
  "created_by": "<username>@<domain>.com",
  "created_on": 163788384993,
  "description": "prestissimo engine for running sql queries",
  "engine_display_name": "sampleEngine",
  "engine_id": "sampleEngine123",
  "external_host_name": "your-hostname.apps.your-domain.com",
  "group_id": "new_group_id",
  "host_name": "your-hostname.apps.your-domain.com",
  "origin": "native",
  "region": "us-south",
  "size_config": "starter",
  "status": "running",
  "status_code": 0,
  "tags": [
    "tag1",
    "tag2"
  ],
  "version": "1.2.0",
  "worker": {
    "node_type": "worker",
    "quantity": 1
  }
}

Get prestissimo engine

Get details of one prestissimo engine

GET /prestissimo_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.prestissimo_engine.get

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Response

Response Body

PrestissimoEngine

EngineDetail

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "update",
    "delete"
  ],
  "associated_catalogs": [
    "hive_data"
  ],
  "coordinator": {
    "node_type": "worker",
    "quantity": 1
  },
  "created_by": "<username>@<domain>.com",
  "created_on": 163788384993,
  "description": "prestissimo engine for running sql queries",
  "engine_display_name": "sampleEngine",
  "engine_id": "sampleEngine123",
  "external_host_name": "your-hostname.apps.your-domain.com",
  "group_id": "new_group_id",
  "host_name": "your-hostname.apps.your-domain.com",
  "origin": "native",
  "region": "us-south",
  "size_config": "starter",
  "status": "running",
  "status_code": 0,
  "tags": [
    "tag1",
    "tag2"
  ],
  "version": "1.2.0",
  "worker": {
    "node_type": "worker",
    "quantity": 1
  }
}

Delete prestissimo engine

Delete a prestissimo engine

DELETE /prestissimo_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.prestissimo_engine.delete

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Response

Status Code

  • 204

    No Content

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

No Sample Response

This method does not specify any sample responses.

Update prestissimo engine

Update details of prestissimo engine

PATCH /prestissimo_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.prestissimo_engine.update

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Request Body

Required*
application/merge-patch+jsonPrestissimoEnginePatch

Update prestissimo engine body

  • curl -X 'PATCH' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/prestissimo_engines/{engine_id}' -H 'accept: application/json' -H 'Content-Type: application/merge-patch+json' -H 'AuthInstanceId: {instance_id}' -d '{
  "description": "updated description for prestissimo engine",
  "engine_display_name": "sample-display-name",
  "engine_properties": {
    "configuration": {
      "coordinator": {
        "property": "value"
      },
      "worker": {
        "property": "value"
      }
    },
    "jvm": {
      "coordinator": {
        "property": "value"
      }
    },
    "catalog": {
      "hive_data": {
        "coordinator": {
          "property": "value"
        },
        "worker": {
          "property": "value"
        }
      }
    },
    "velox": {
      "property": "value"
    },
    "log_config": {
      "coordinator": {
        "property": "value"
      },
      "worker": {
        "property": "value"
      }
    },
    "global": {
      "property": "value"
    }
  },
  "engine_restart": "force",
  "remove_engine_properties": {
    "configuration": {
      "coordinator": [
        "property"
      ],
      "worker": [
        "property"
      ]
    },
    "catalog": {
      "hive_data": {
        "coordinator": [
          "property"
        ],
        "worker": [
          "property"
        ]
      }
    },
    "jvm": {
      "coordinator": [
        "property"
      ]
    },
    "log_config": {
      "coordinator": [
        "property"
      ],
      "worker": [
        "property"
      ]
    },
    "velox": [
      "property"
    ]
  },
  "tags": [
    "tag1",
    "tag2"
  ]
}'

Response

Response Body

PrestissimoEngine

EngineDetail

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "update",
    "delete"
  ],
  "associated_catalogs": [
    "hive_data"
  ],
  "coordinator": {
    "node_type": "worker",
    "quantity": 1
  },
  "created_by": "<username>@<domain>.com",
  "created_on": 163788384993,
  "description": "prestissimo engine for running sql queries",
  "engine_display_name": "sampleEngine",
  "engine_id": "sampleEngine123",
  "external_host_name": "your-hostname.apps.your-domain.com",
  "group_id": "new_group_id",
  "host_name": "your-hostname.apps.your-domain.com",
  "origin": "native",
  "region": "us-south",
  "size_config": "starter",
  "status": "running",
  "status_code": 0,
  "tags": [
    "tag1",
    "tag2"
  ],
  "version": "1.2.0",
  "worker": {
    "node_type": "worker",
    "quantity": 1
  }
}

Get prestissimo engine catalogs

Get list of all catalogs attached a prestissimo engine

GET /prestissimo_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.prestissimo_engine_catalog.list

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Response

Response Body

CatalogCollection

GetCatalogs OK

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "catalogs": []
}

Associate catalogs to a prestissimo engine

Associate catalogs to a prestissimo engine

POST /prestissimo_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.prestissimo_engine_catalog.add

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Request Body

Required*
CatalogPrototype

Request body

Examples: 
{
  "catalog_names": "iceberg_data, catalog02"
}
  • curl -X 'POST' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/prestissimo_engines/{engine_id}/catalogs' -H 'accept: application/json' -H "Content-Type: application/json" -d '{ "catalog_names": "catalog_name01,catalog_name02" }' -H "AuthInstanceId: {instance_id}"

Response

Response Body

Catalog

Define the catalog details

Status Code

  • 201

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "view",
    "update",
    "delete"
  ],
  "associated_buckets": [
    "ibm_cos_bucket"
  ],
  "associated_databases": [
    "iceberg_data"
  ],
  "associated_engines": [
    "presto367"
  ],
  "catalog_name": "iceberg_data",
  "catalog_type": "iceberg",
  "created_by": "user@domain.com",
  "created_on": "1700633239",
  "description": "catalog description",
  "hostname": "1234-xyz456-abc4321.lakehouse.dev.appdomain.cloud",
  "port": "31504",
  "sync_exception": [
    "table is corrupted",
    "table metadata not there"
  ],
  "tags": [
    "tag1",
    "tag2"
  ],
  "thrift_uri": "thrift://samplehost-catalog:4354"
}

Disassociate catalogs from a prestissimo engine

Disassociate one or more catalogs from a prestissimo engine

DELETE /prestissimo_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.prestissimo_engine_catalog.remove

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Query Parameters

  • catalog_names
    Required*
    string

    Catalog id(s) to be stopped, comma separated

    Possible values: 1 ≤ length ≤ 1000, Value must match regular expression ^[a-zA-Z0-9\-,_]+$

Response

Status Code

  • 204

    No Content

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

No Sample Response

This method does not specify any sample responses.

Get prestissimo engine catalog

Get catalog attached to a prestissimo engine

GET /prestissimo_engines/{engine_id}/catalogs/{catalog_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.prestissimo_engine_catalog.get

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • catalog_id
    Required*
    string

    catalog id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-_]+$

Response

Response Body

Catalog

Define the catalog details

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "view",
    "update",
    "delete"
  ],
  "associated_buckets": [
    "ibm_cos_bucket"
  ],
  "associated_databases": [
    "iceberg_data"
  ],
  "associated_engines": [
    "presto367"
  ],
  "catalog_name": "iceberg_data",
  "catalog_type": "iceberg",
  "created_by": "user@domain.com",
  "created_on": "1700633239",
  "description": "catalog description",
  "hostname": "1234-xyz456-abc4321.lakehouse.dev.appdomain.cloud",
  "port": "31504",
  "sync_exception": [
    "table is corrupted",
    "table metadata not there"
  ],
  "tags": [
    "tag1",
    "tag2"
  ],
  "thrift_uri": "thrift://samplehost-catalog:4354"
}

Explain query

Explain a query statement

POST /prestissimo_engines/{engine_id}/query_explain

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.query_explain.create

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    Engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Request Body

Required*
ExplainPrototype

Request body

Response

Response Body

ResultPrestissimoExplainStatement

ExplainStatement OK

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "result": "explain statement"
}

Explain analyze

Return query metrics after query is complete

POST /prestissimo_engines/{engine_id}/query_explain_analyze

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.query_explain_analyze.create

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    Engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Request Body

Required*
ExplainAnalyzePrototype

Request body

Response

Response Body

ResultRunPrestissimoExplainAnalyzeStatement

explainAnalyzeStatement OK

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "result": "explain statement"
}

Restart a prestissimo engine

Restart an existing prestissimo engine

POST /prestissimo_engines/{engine_id}/restart

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.prestissimo_engine.refresh

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Response

Response Body

SuccessResponse

Response of success

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "message": "Successful message",
  "message_code": "success"
}

Get list of presto engines

Get list of all presto engines

GET /presto_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.presto_engine.list

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

  • curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/presto_engines"

Response

Response Body

PrestoEngineCollection

List Presto engines

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "presto_engines": [
    {
      "actions": [
        "view",
        "use",
        "update",
        "select",
        "access_ui",
        "associate",
        "disassociate",
        "restart",
        "pause",
        "resume",
        "grant",
        "revoke",
        "delete",
        "create",
        "scale"
      ],
      "associated_catalogs": [
        "iceberg_data",
        "hive_data"
      ],
      "build_version": "1.1.0.0.0",
      "coordinator": {
        "node_type": "bx2.4x16",
        "quantity": 1
      },
      "engine_display_name": "starter",
      "engine_id": "presto511",
      "external_host_name": "your-hostname.apps.your-domain.com",
      "group_id": "presto511",
      "host_name": "1234-xyz456-abc4321.databases.appdomain.cloud",
      "origin": "native",
      "port": 30156,
      "size_config": "starter",
      "status": "running",
      "status_code": 0,
      "tags": [
        "tag1",
        "tag2"
      ],
      "type": "presto",
      "version": "v0.282",
      "worker": {
        "node_type": "bx2.4x16",
        "quantity": 1
      }
    }
  ]
}

Create presto engine

Create a new presto engine

POST /presto_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.presto_engine.create

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*
PrestoEnginePrototype

Create Engine Body

  • curl -X 'POST' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/presto_engines' -H 'accept: application/json' -H 'Content-Type: application/json' -d '{ 
  "description": "", 
  "engine_details": { 
    "coordinator": { 
      "node_type": "starter", 
      "quantity": 1 
    }, 
    "size_config": "custom", 
    "worker": { 
      "node_type": "starter", 
      "quantity": 1 
    } 
  }, 
  "engine_display_name": "presto572", 
  "origin": "native", 
  "associated_catalogs": [], 
  "version": "v0.286", 
  "tags": [], 
  "region": "" 
}'

Response

Response Body

PrestoEngine

EngineDetail

Status Code

  • 201

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "view",
    "use",
    "update",
    "select",
    "access_ui",
    "associate",
    "disassociate",
    "restart",
    "pause",
    "resume",
    "grant",
    "revoke",
    "delete",
    "create",
    "scale"
  ],
  "associated_catalogs": [
    "iceberg_data",
    "hive_data"
  ],
  "build_version": "1.1.0.0.0",
  "coordinator": {
    "node_type": "bx2.4x16",
    "quantity": 1
  },
  "engine_display_name": "starter",
  "engine_id": "presto511",
  "external_host_name": "your-hostname.apps.your-domain.com",
  "group_id": "presto511",
  "host_name": "1234-xyz456-abc4321.databases.appdomain.cloud",
  "origin": "native",
  "port": 30156,
  "size_config": "starter",
  "status": "running",
  "status_code": 0,
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "presto",
  "version": "v0.282",
  "worker": {
    "node_type": "bx2.4x16",
    "quantity": 1
  }
}

Get presto engine

Get details of one presto engine

GET /presto_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.presto_engine.get

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}' -H 'accept: application/json' -H "AuthInstanceId: {instance id}"

Response

Response Body

PrestoEngine

EngineDetail

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "view",
    "use",
    "update",
    "select",
    "access_ui",
    "associate",
    "disassociate",
    "restart",
    "pause",
    "resume",
    "grant",
    "revoke",
    "delete",
    "create",
    "scale"
  ],
  "associated_catalogs": [
    "iceberg_data",
    "hive_data"
  ],
  "build_version": "1.1.0.0.0",
  "coordinator": {
    "node_type": "bx2.4x16",
    "quantity": 1
  },
  "engine_display_name": "starter",
  "engine_id": "presto511",
  "external_host_name": "your-hostname.apps.your-domain.com",
  "group_id": "presto511",
  "host_name": "1234-xyz456-abc4321.databases.appdomain.cloud",
  "origin": "native",
  "port": 30156,
  "size_config": "starter",
  "status": "running",
  "status_code": 0,
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "presto",
  "version": "v0.282",
  "worker": {
    "node_type": "bx2.4x16",
    "quantity": 1
  }
}

Delete presto engine

Delete a presto engine

DELETE /presto_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.presto_engine.delete

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X 'DELETE' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}' -H 'accept: */*' -H "AuthInstanceId: {instance id}"

Response

Status Code

  • 204

    No Content

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

No Sample Response

This method does not specify any sample responses.

Update presto engine

Update details of presto engine

PATCH /presto_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.presto_engine.update

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Request Body

Required*
application/merge-patch+jsonPrestoEnginePatch

Update Engine Body

  • curl -X 'PATCH' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}' -H 'accept: application/json' -H 'LhInstanceId: {InstanceID}' -H 'Content-Type: application/json' -d '{
  "description": "updated description for presto engine",
  "engine_display_name": "engine-display-name",
  "engine_properties": {
    "configuration": {
      "coordinator": {
        "property": "value"
      },
      "worker": {
        "property": "value"
      }
    },
    "jvm": {
      "coordinator": {
        "property": "value"
      },
      "worker": {
        "property": "value"
      }
    },
    "catalog": {
      "catalog_name": {
        "coordinator": {
          "property": "value"
        },
        "worker": {
          "property": "value"
        }
      }
    },
    "event_listener": {
      "property": "value"
    },
    "jmx_exporter_config": {
      "property": "value"
    },
    "log_config": {
      "coordinator": {
        "property": "value"
      },
      "worker": {
        "property": "value"
      }
    },
    "global": {
      "enable-mixed-case-support": "true"
    }
  },
  "engine_restart": "force",
  "remove_engine_properties": {
    "configuration": {
      "coordinator": [
        "property"
      ],
      "worker": [
        "property"
      ]
    },
    "jvm": {
      "coordinator": [
        "property"
      ],
      "worker": [
        "property"
      ]
    },
    "log_config": {
      "coordinator": [
        "property"
      ],
      "worker": [
        "property"
      ]
    },
    "event_listener": [
      "property"
    ],
    "jmx_exporter_config": [
      "property"
    ],
    "global": [
      "property"
    ],
    "catalog": {}
  },
  "tags": [
    "tag1",
    "tag2"
  ]
}'

Response

Response Body

PrestoEngine

EngineDetail

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "view",
    "use",
    "update",
    "select",
    "access_ui",
    "associate",
    "disassociate",
    "restart",
    "pause",
    "resume",
    "grant",
    "revoke",
    "delete",
    "create",
    "scale"
  ],
  "associated_catalogs": [
    "iceberg_data",
    "hive_data"
  ],
  "build_version": "1.1.0.0.0",
  "coordinator": {
    "node_type": "bx2.4x16",
    "quantity": 1
  },
  "engine_display_name": "starter",
  "engine_id": "presto511",
  "external_host_name": "your-hostname.apps.your-domain.com",
  "group_id": "presto511",
  "host_name": "1234-xyz456-abc4321.databases.appdomain.cloud",
  "origin": "native",
  "port": 30156,
  "size_config": "starter",
  "status": "running",
  "status_code": 0,
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "presto",
  "version": "v0.282",
  "worker": {
    "node_type": "bx2.4x16",
    "quantity": 1
  }
}

Get presto engine catalogs

Get list of all catalogs attached to a presto engine

GET /presto_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.presto_engine_catalog.list

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/catalogs' -H 'accept: application/json' -H "AuthInstanceId: {instance id}"

Response

Response Body

CatalogCollection

GetCatalogs OK

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "catalogs": []
}

Associate catalogs to presto engine

Associate one or more catalogs to a presto engine

POST /presto_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.presto_engine_catalog.add

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Request Body

Required*
CatalogPrototype

Request body

Examples: 
{
  "catalog_names": "iceberg_data, catalog02"
}
  • curl -X 'POST' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/catalogs' -H 'accept: application/json' -H "Content-Type: application/json" -d '{ "catalog_names":  "catalog_name01,catalog_name02" }' -H "AuthInstanceId: {instance_id}"

Response

Response Body

Catalog

Define the catalog details

Status Code

  • 201

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "view",
    "update",
    "delete"
  ],
  "associated_buckets": [
    "ibm_cos_bucket"
  ],
  "associated_databases": [
    "iceberg_data"
  ],
  "associated_engines": [
    "presto367"
  ],
  "catalog_name": "iceberg_data",
  "catalog_type": "iceberg",
  "created_by": "user@domain.com",
  "created_on": "1700633239",
  "description": "catalog description",
  "hostname": "1234-xyz456-abc4321.lakehouse.dev.appdomain.cloud",
  "port": "31504",
  "sync_exception": [
    "table is corrupted",
    "table metadata not there"
  ],
  "tags": [
    "tag1",
    "tag2"
  ],
  "thrift_uri": "thrift://samplehost-catalog:4354"
}

Disassociate catalogs from a presto engine

Disassociate one or more catalogs from a presto engine

DELETE /presto_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.presto_engine_catalog.remove

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Query Parameters

  • catalog_names
    Required*
    string

    Catalog id(s) to be stopped, comma separated

    Possible values: 1 ≤ length ≤ 1000, Value must match regular expression ^[a-zA-Z0-9\-_,]+$

  • curl -X 'DELETE' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/catalogs?catalog_names=catalog_name1' -H 'accept: */*' -H "AuthInstanceId: {instance id}"

Response

Status Code

  • 204

    No Content

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

No Sample Response

This method does not specify any sample responses.

Get presto engine catalog

Get catalog attached to presto engine

GET /presto_engines/{engine_id}/catalogs/{catalog_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.presto_engine_catalog.get

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • catalog_id
    Required*
    string

    catalog id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-_]+$

  • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/catalogs/{catalog_id}' -H 'accept: application/json' -H "AuthInstanceId: {instance id}"

Response

Response Body

Catalog

Define the catalog details

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "view",
    "update",
    "delete"
  ],
  "associated_buckets": [
    "ibm_cos_bucket"
  ],
  "associated_databases": [
    "iceberg_data"
  ],
  "associated_engines": [
    "presto367"
  ],
  "catalog_name": "iceberg_data",
  "catalog_type": "iceberg",
  "created_by": "user@domain.com",
  "created_on": "1700633239",
  "description": "catalog description",
  "hostname": "1234-xyz456-abc4321.lakehouse.dev.appdomain.cloud",
  "port": "31504",
  "sync_exception": [
    "table is corrupted",
    "table metadata not there"
  ],
  "tags": [
    "tag1",
    "tag2"
  ],
  "thrift_uri": "thrift://samplehost-catalog:4354"
}

Explain presto query

Explain a query statement

POST /presto_engines/{engine_id}/query_explain

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.query_explain.create

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    Engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Request Body

Required*
ExplainPrototype

Request body

  • curl -X 'POST' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/query_explain' -H 'accept: application/json' -H 'Content-Type: application/json' -H "AuthInstanceId: {instance id}" -d '{  "format": "json",  "statement": "show schemas in catalog_name",  "type": "io"}'

Response

Response Body

RunExplainStatementOKBody

ExplainStatement OK

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "response": {
    "message": "explain statement",
    "message_code": "success"
  },
  "result": "explain statement"
}

Explain presto analyze

Return query metrics after query is complete

POST /presto_engines/{engine_id}/query_explain_analyze

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.query_explain_analyze.create

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    Engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Request Body

Required*
ExplainAnalyzePrototype

Request body

  • curl -X 'POST' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/query_explain_analyze' -H 'accept: application/json' -H 'Content-Type: application/json' -H "AuthInstanceId: {instance id}" -d '{  "statement": "show schemas in catalog_name",  "verbose": true}'

Response

Response Body

RunExplainAnalyzeStatementOKBody

explainAnalyzeStatement OK

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "response": {
    "message": "explain presto analyze",
    "message_code": "success"
  },
  "result": "explain statement"
}

Restart a presto engine

Restart an existing presto engine

POST /presto_engines/{engine_id}/restart

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.presto_engine.refresh

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X 'POST' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/presto_engines/{engine_id}/restart' -H 'accept: application/json' -H "AuthInstanceId: {instance id}" -d ''

Response

Response Body

CreateEngineRestartCreatedBody

restart engine

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "response": {
    "message": "Restart presto engine",
    "message_code": "success"
  }
}

List all spark engines

List all spark engines

GET /spark_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_engine.list

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

  • curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines"

Response

Response Body

SparkEngineCollection

List spark engines

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "spark_engines": [
    {
      "actions": [
        "action1",
        "action2",
        "action3"
      ],
      "associated_catalogs": [],
      "created_by": "user@example.com",
      "created_on": 1234567890,
      "description": "Sample Description",
      "engine_details": {
        "history_server_endpoint": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_api/v1",
        "spark_access_endpoint": "https://console.example.com/analytics-engine/details/spark-1717414394213457",
        "spark_jobs_v4_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/spark_applications",
        "spark_kernel_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/jkg/api/kernels",
        "view_history_server": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_ui",
        "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/abcd-efgh-ijkl-mnop/spark_engines/spark119/applications"
      },
      "engine_display_name": "sample-external-spark-co-located",
      "engine_id": "spark456",
      "origin": "external",
      "status": "REGISTERED",
      "tags": [
        "tag1",
        "tag2"
      ],
      "type": "spark"
    },
    {
      "actions": [
        "action1",
        "action2",
        "action3"
      ],
      "associated_catalogs": [
        "iceberg_data",
        "hudi_catalog",
        "delta_lake"
      ],
      "created_by": "user@example.com",
      "created_on": 1234567890,
      "description": "Sample Description",
      "engine_details": {
        "default_config": {
          "config1": "value1",
          "config2": "value2"
        },
        "default_version": "3.3",
        "endpoints": {
          "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/spark_engines/spark123/applications",
          "wxd_engine_endpoint": "https://example.com/lakehouse/api/v2/spark_engines/spark123",
          "wxd_history_server_endpoint": "https://example.com/lakehouse/api/v2/spark_engines/spark123/history_server",
          "wxd_history_server_ui_endpoint": "https://example.com/lakehouse/api/v2/1234567890123456/spark_engines/spark123/history_server/ui"
        },
        "engine_home_volume": "cpd-instance::volume-name"
      },
      "engine_display_name": "sample-native-spark",
      "engine_id": "spark123",
      "origin": "native",
      "status": "running",
      "tags": [],
      "type": "spark"
    }
  ]
}

Create spark engine

Create a new spark engine

POST /spark_engines

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_engine.create

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Request Body

Required*
SparkEnginePrototype

Create spark engine Body

Examples: 
{
  "description": "",
  "engine_details": {
    "default_version": "3.4",
    "engine_home_volume_storage_size": "5Gi",
    "engine_home_volume_name": "volume-name",
    "engine_home_volume_storage_class": "nfs-client"
  },
  "engine_display_name": "sample-native-spark-new-volume",
  "type": "spark",
  "associated_catalogs": [
    "iceberg_data",
    "hive_data"
  ],
  "origin": "native"
}
{
  "description": "",
  "engine_details": {
    "default_version": "3.3",
    "engine_home_volume_storage_size": "5Gi",
    "engine_home_volume_name": "volume-name",
    "engine_home_volume_storage_class": "nfs-client"
  },
  "engine_display_name": "sample-native-spark-new-volume",
  "associated_catalogs": [
    "iceberg_data",
    "hive_data"
  ],
  "type": "spark",
  "origin": "native"
}
{
  "description": "",
  "engine_details": {
    "instance_id": "1716437041193635"
  },
  "engine_display_name": "sample-external-spark-co-located",
  "origin": "discover"
}
{
  "description": "",
  "engine_details": {
    "connection_string": "https://<host>/<version>/analytics_engines/<instance_id>",
    "api_key": "api_Key",
    "managed_by": "fully"
  },
  "engine_display_name": "sample-external-spark-fully-managed",
  "origin": "external"
}
{
  "description": "",
  "engine_details": {
    "connection_string": "hhttps://<host>/<version>/analytics_engines/<instance_id>",
    "api_key": "zen_api_Key",
    "managed_by": "self"
  },
  "engine_display_name": "sample-external-spark-self-managed",
  "origin": "external"
}
  • curl -X POST -H "content-type: application/json" -H "accept: application/json" -d '{ 
  "description": "Spark engine", 
  "engine_details": { 
    "default_version": "3.5", 
    "scale_config": { 
      "node_type": "small", 
      "number_of_nodes": 1 
    }, 
    "engine_home_bucket_name": "wxd-bucket-u2z35rryy45rov8" 
  }, 
  "engine_display_name": "spark-D01", 
  "origin": "native", 
  "type": "spark", 
  "associated_catalogs": [ 
    "iceberg_data" 
  ] 
}' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines"
  • curl -X POST -H "content-type: application/json" -H "accept: application/json" -d '{ 
  "description": "", 
  "engine_details": { 
    "default_version": "3.3", 
    "engine_home_volume_storage_size": "5Gi", 
    "engine_home_volume_name": "volume-name", 
    "engine_home_volume_storage_class": "nfs-client" 
  }, 
  "engine_display_name": "sample-native-spark-new-volume", 
  "associated_catalogs": [ 
    "iceberg_data", 
    "hive_data" 
  ], 
  "type": "spark", 
  "origin": "native" 
}' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines"
  • curl -X POST -H "content-type: application/json" -H "accept: application/json" -d '{ 
  "description": "", 
  "engine_details": { 
    "default_version": "3.3", 
    "engine_home_volume_storage_size": "5Gi", 
    "engine_home_volume_name": "volume-name", 
    "engine_home_volume_storage_class": "nfs-client" 
  }, 
  "engine_display_name": "sample-native-spark-new-volume", 
  "associated_catalogs": [ 
    "iceberg_data", 
    "hive_data" 
  ], 
  "type": "gluten", 
  "origin": "native" 
}' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines"
  • curl -X POST -H "content-type: application/json" -H "accept: application/json" -d '{ 
  "description": "", 
  "engine_details": { 
    "instance_id": "1716437041193635" 
  }, 
  "engine_display_name": "sample-external-spark-co-located", 
  "origin": "discover" 
}'  "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines"
  • curl -X POST -H "content-type: application/json" -H "accept: application/json" -d '{ 
  "description": "", 
  "engine_details": { 
    "connection_string": "https://<host>/<version>/analytics_engines/<instance_id>", 
    "api_key": "api_Key", 
    "managed_by": "fully" 
  }, 
  "engine_display_name": "sample-external-spark-fully-managed", 
  "origin": "external" 
}'  "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines"
  • curl -X POST -H "content-type: application/json" -H "accept: application/json" -d '{"description": "","engine_details": {"connection_string": "hhttps://<host>/<version>/analytics_engines/<instance_id>","api_key": "zen_api_Key","managed_by": "self"},"engine_display_name": "sample-external-spark-self-managed","origin": "external"}'  "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines"

Response

Response Body

SparkEngine

EngineDetail

Status Code

  • 202

    Accepted

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [],
  "associated_catalogs": [
    "iceberg_data"
  ],
  "engine_id": "spark123",
  "origin": "native",
  "status": "runnung",
  "tags": [
    "tag1",
    "tag2"
  ]
}
  • {
  "actions": [],
  "associated_catalogs": [],
  "engine_id": "spark530",
  "origin": "discover",
  "status": "registered",
  "tags": [
    "tag1",
    "tag2"
  ]
}
  • {
  "actions": [],
  "associated_catalogs": [],
  "engine_id": "spark810",
  "origin": "external",
  "status": "",
  "tags": [
    "tag1",
    "tag2"
  ]
}
  • {
  "actions": [],
  "associated_catalogs": [],
  "engine_id": "spark728",
  "origin": "external",
  "status": "registered",
  "tags": [
    "tag1",
    "tag2"
  ]
}

Get spark engine

Get spark engine by ID

GET /spark_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_engine.get

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X GET -H "accept: */*" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}"

Response

Response Body

SparkEngine

EngineDetail

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "view",
    "use",
    "update",
    "select",
    "access_ui",
    "associate",
    "disassociate",
    "restart",
    "pause",
    "resume",
    "grant",
    "revoke",
    "delete",
    "create",
    "scale"
  ],
  "associated_catalogs": [],
  "created_by": "cpadmin",
  "created_on": 1749365970,
  "description": "",
  "engine_details": {
    "default_config": {
      "spark.hadoop.wxd.cpg.endpoint": "https://ibm-lh-lakehouse-cpg-svc.cpd-instance.svc.cluster.local:8680"
    },
    "default_version": "3.4",
    "endpoints": {
      "wxd_application_endpoint": "https://cpd-cpd-instance.apps.wxd-gluten-35.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/spark758/applications",
      "wxd_engine_endpoint": "https://cpd-cpd-instance.apps.wxd-gluten-35.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/spark758",
      "wxd_history_server_endpoint": "https://cpd-cpd-instance.apps.wxd-gluten-35.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/spark758/history_server",
      "wxd_history_server_ui_endpoint": "https://cpd-cpd-instance.apps.wxd-gluten-35.cp.fyre.ibm.com/lakehouse/api/v2/1749315427957202/spark_engines/spark758/history_server/ui"
    },
    "engine_home_volume": "cpd-instance::test-gluten",
    "vscode_config": {
      "crn": "",
      "environment_type": "Software",
      "host": "cpd-cpd-instance.apps.wxd-gluten-35.cp.fyre.ibm.com",
      "user_name": "cpadmin"
    }
  },
  "engine_display_name": "test-gluten-35",
  "engine_id": "spark758",
  "origin": "native",
  "status": "running",
  "tags": null,
  "type": "gluten"
}
  • {
  "actions": [
    "action1",
    "action2",
    "action3"
  ],
  "associated_catalogs": [],
  "created_by": "user@example.com",
  "created_on": 1234567890,
  "description": "Sample Description",
  "engine_details": {
    "history_server_endpoint": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_api/v1",
    "spark_access_endpoint": "https://console.example.com/analytics-engine/details/spark-1234567890123456",
    "spark_jobs_v4_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/spark_applications",
    "spark_kernel_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/jkg/api/kernels",
    "view_history_server": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_ui",
    "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/abcd-efgh-ijkl-mnop/spark_engines/spark119/applications"
  },
  "engine_display_name": "sample-external-spark-co-located",
  "engine_id": "spark456",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "spark"
}
  • {
  "actions": [
    "action1",
    "action2",
    "action3"
  ],
  "associated_catalogs": [],
  "created_by": "user@example.com",
  "created_on": 1234567890,
  "description": "Sample Description",
  "engine_details": {
    "applications_api": "https://api.example.com/v3/analytics_engines/abcd-efgh-ijkl-mnop/applications",
    "history_server_endpoint": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_api/v1",
    "view_history_server": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_ui",
    "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/abcd-efgh-ijkl-mnop/spark_engines/spark119/applications"
  },
  "engine_display_name": "sample-external-spark-fully-managed",
  "engine_id": "spark456",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "spark"
}
  • {
  "actions": [
    "action1",
    "action2",
    "action3"
  ],
  "associated_catalogs": [],
  "created_by": "user@example.com",
  "created_on": 1234567890,
  "description": "Sample Description",
  "engine_details": {
    "connection_string": "https://api.example.com/v3/analytics_engines/abcd-efgh-ijkl-mnop",
    "endpoints": {
      "history_server_endpoint": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_api/v1",
      "spark_access_endpoint": "https://console.example.com/analytics-engine/details/spark-1234567890123456",
      "spark_jobs_v4_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/spark_applications",
      "spark_kernel_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/jkg/api/kernels",
      "view_history_server": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_ui",
      "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/abcd-efgh-ijkl-mnop/spark_engines/spark119/applications"
    }
  },
  "engine_display_name": "sample-external-spark-self-managed",
  "engine_id": "spark456",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "spark"
}

Delete spark engine

Delete a spark engine

DELETE /spark_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_engine.delete

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X DELETE -H "accept: */*" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}"

Response

Status Code

  • 204

    No Content

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

No Sample Response

This method does not specify any sample responses.

Update spark engine

Update details of spark engine

PATCH /spark_engines/{engine_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_engine.update

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Request Body

Required*
application/merge-patch+jsonUpdateSparkEngineBody

Update Engine Body

Examples: 
{
  "engine_display_name": "Updated Display Name",
  "description": "Updated Description",
  "tags": [
    "tag1",
    "tag2"
  ],
  "engine_details": {
    "default_config": {
      "config1": "value1",
      "config2": "value2"
    },
    "default_version": "3.4",
    "resource_limits": {
      "cores": "No. of CPU limit",
      "memory": "Amount of memory limit in GBs"
    },
    "resource_limit_enabled": true
  }
}
{
  "engine_display_name": "Updated Display Name",
  "description": "Updated description ",
  "tags": [
    "tag1"
  ]
}
  • curl -X PATCH -H "content-type: application/json" -H "accept: application/json" -d '{ "description": "Updated Description", "engine_details": { "default_config": { "config1": "value1", "config2": "value2" }, "default_version": "3.4" }, "engine_display_name": "Updated Display Name", "tags": [ "tag1", "tag2" ] }' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}"
  • curl -X PATCH -H "content-type: application/json" -H "accept: application/json" -d '{ "description": "Updated description", "engine_display_name": "Updated Display Name", "tags": [ "tag1" ] }' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}"

Response

Response Body

SparkEngine

EngineDetail

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [
    "action1",
    "action2",
    "action3"
  ],
  "associated_catalogs": [
    "iceberg_data",
    "hudi_catalog",
    "delta_lake"
  ],
  "created_by": "user@example.com",
  "created_on": 1234567890,
  "description": "Sample Description",
  "engine_details": {
    "default_config": {
      "config1": "value1",
      "config2": "value2"
    },
    "default_version": "3.3",
    "endpoints": {
      "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/spark_engines/spark123/applications",
      "wxd_engine_endpoint": "https://example.com/lakehouse/api/v2/spark_engines/spark123",
      "wxd_history_server_endpoint": "https://example.com/lakehouse/api/v2/spark_engines/spark123/history_server",
      "wxd_history_server_ui_endpoint": "https://example.com/lakehouse/api/v2/1234567890123456/spark_engines/spark123/history_server/ui"
    },
    "engine_home_volume": "cpd-instance::volume-name"
  },
  "engine_display_name": "sample-narive-spark",
  "engine_id": "spark123",
  "origin": "native",
  "status": "running",
  "tags": [],
  "type": "spark"
}
  • {
  "actions": [
    "action1",
    "action2",
    "action3"
  ],
  "associated_catalogs": [],
  "created_by": "user@example.com",
  "created_on": 1234567890,
  "description": "Sample Description",
  "engine_details": {
    "history_server_endpoint": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_api/v1",
    "spark_access_endpoint": "https://console.example.com/analytics-engine/details/spark-1234567890123456",
    "spark_jobs_v4_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/spark_applications",
    "spark_kernel_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/jkg/api/kernels",
    "view_history_server": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_ui",
    "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/abcd-efgh-ijkl-mnop/spark_engines/spark119/applications"
  },
  "engine_display_name": "sample-external-spark-co-located",
  "engine_id": "spark456",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "spark"
}
  • {
  "actions": [
    "action1",
    "action2",
    "action3"
  ],
  "associated_catalogs": [],
  "created_by": "user@example.com",
  "created_on": 1234567890,
  "description": "Sample Description",
  "engine_details": {
    "applications_api": "https://api.example.com/v3/analytics_engines/abcd-efgh-ijkl-mnop/applications",
    "history_server_endpoint": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_api/v1",
    "view_history_server": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_ui",
    "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/abcd-efgh-ijkl-mnop/spark_engines/spark119/applications"
  },
  "engine_display_name": "sample-external-spark-fully-managed",
  "engine_id": "spark456",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "spark"
}
  • {
  "actions": [
    "action1",
    "action2",
    "action3"
  ],
  "associated_catalogs": [],
  "created_by": "user@example.com",
  "created_on": 1234567890,
  "description": "Sample Description",
  "engine_details": {
    "connection_string": "https://api.example.com/v3/analytics_engines/abcd-efgh-ijkl-mnop",
    "endpoints": {
      "history_server_endpoint": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_api/v1",
      "spark_access_endpoint": "https://console.example.com/analytics-engine/details/spark-1234567890123456",
      "spark_jobs_v4_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/spark_applications",
      "spark_kernel_endpoint": "https://console.example.com/v4/analytics_engines/abcd-efgh-ijkl-mnop/jkg/api/kernels",
      "view_history_server": "https://console.example.com/v3/analytic_engines/abcd-efgh-ijkl-mnop/spark_history_ui",
      "wxd_application_endpoint": "https://example.com/lakehouse/api/v2/abcd-efgh-ijkl-mnop/spark_engines/spark119/applications"
    }
  },
  "engine_display_name": "sample-external-spark-self-managed",
  "engine_id": "spark456",
  "origin": "external",
  "status": "REGISTERED",
  "tags": [
    "tag1",
    "tag2"
  ],
  "type": "spark"
}

List all applications in a spark engine

List all applications in a spark engine

GET /spark_engines/{engine_id}/applications

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_engine_application.list

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Query Parameters

  • state
    string[]

    state

    Possible values: 0 ≤ number of items ≤ 100, 1 ≤ length ≤ 128, Value must match regular expression ^.*$

  • curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/applications"

Response

Response Body

SparkEngineApplicationStatusCollection

Engine Application Detail

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "applications": [
    {
      "application_id": "93b1a765-e3bc-4972-acc4-5b3d1a0f5d62",
      "auto_termination_time": "2024-05-15T18:11:54.615Z",
      "id": "93b1a765-e3bc-4972-acc4-5b3d1a0f5d62",
      "spark_application_id": "app-20240415181154-0000",
      "spark_application_name": "auto-scale-test",
      "spark_version": "3.4",
      "start_time": "2024-04-15T18:11:54.615Z",
      "state": "running",
      "submission_time": "2024-04-15T18:11:21.458Z"
    },
    {
      "application_id": "b2d3f4g5-h6i7-j8k9-l0m1-n2o3p4q5r6s7",
      "auto_termination_time": "2025-06-16T19:12:55.716Z",
      "id": "b2d3f4g5-h6i7-j8k9-l0m1-n2o3p4q5r6s7",
      "spark_application_id": "app-20250516191255-0001",
      "spark_application_name": "data-processing-test",
      "spark_version": "3.3",
      "start_time": "2025-05-16T19:12:55.716Z",
      "state": "finished",
      "submission_time": "2025-05-16T19:12:22.559Z"
    }
  ]
}
  • {
  "applications": [
    {
      "application_id": "abcd-efgh-5806ab96-a5ba-4488-871b-162e13f6bf90",
      "id": "abcd-efgh-5806ab96-a5ba-4488-871b-162e13f6bf90",
      "runtime": {
        "spark_version": "3.3"
      },
      "state": "accepted",
      "submission_time": "2024-04-15T18:13:17.507Z"
    },
    {
      "application_id": "abcd-efgh-0969c82b-2980-48f4-8896-800e311a4696",
      "auto_termination_time": "2024-03-25T15:57:06.762Z",
      "end_time": "2024-03-22T15:57:24.703Z",
      "finish_time": "2024-03-22T15:57:24.703Z",
      "id": "abcd-efgh-0969c82b-2980-48f4-8896-800e311a4696",
      "runtime": {
        "spark_version": "3.3"
      },
      "spark_application_id": "app-20240322155706-0000",
      "spark_application_name": "PythonWordCount",
      "start_time": "2024-03-22T15:57:06.762Z",
      "state": "finished",
      "submission_time": "2024-03-22T15:56:41.400Z"
    }
  ]
}

Submit engine applications

Submit engine applications

POST /spark_engines/{engine_id}/applications

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_engine_application.create

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Query Parameters

  • state
    string[]

    state

    Possible values: 0 ≤ number of items ≤ 100, 1 ≤ length ≤ 128, Value must match regular expression ^.*$

Request Body

Required*
SparkEngineApplicationStatusPrototype

Create spark application parameters

Examples: 
{
  "application_details": {
    "application": "/opt/ibm/spark/examples/src/main/python/wordcount.py",
    "arguments": [
      "/opt/ibm/spark/examples/src/main/resources/people.txt"
    ]
  }
}
  • curl -X POST -H "content-type: application/json" -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -d '{ "application_details": { "application": "s3://mybucket/wordcount.py", "arguments": [ "people.txt" ], "conf": { "additionalProp1": "key:value", "additionalProp2": "key:value", "additionalProp3": "key:value" }, "env": { "additionalProp1": "key:value", "additionalProp2": "key:value", "additionalProp3": "key:value" }, "name": "SparkApplicaton1" }, "job_endpoint": "<host>/v4/analytics_engines/c7b3fccf-badb-46b0-b1ef-9b3154424021/engine_applications", "service_instance_id": "3FGeNUZqcdu6rOFbaHhGBoNdCHj98UoHd6nUFqkfsKhgacYD8ImM6iSA_F-TjHUdEZuVsoQTmOn", "type": "iae" }' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/applications"

Response

Response Body

SparkEngineApplicationStatus

Engine Application Status

Status Code

  • 201

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "id": "2ad2af4b-9747-4652-8e6e-d0a91599f9a2",
  "state": "accepted"
}

Stop Spark Applications

Stop a running spark application

DELETE /spark_engines/{engine_id}/applications

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_engine_application.delete

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Query Parameters

  • application_id
    Required*
    string

    Application id(s) to be stopped, comma separated

    Possible values: 1 ≤ length ≤ 1000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • state
    string[]

    state

    Possible values: 0 ≤ number of items ≤ 100, 1 ≤ length ≤ 128, Value must match regular expression ^.*$

  • curl -X DELETE -H "accept: */*" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/applications?application_id={application_id}"

Response

Status Code

  • 204

    No Content

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

No Sample Response

This method does not specify any sample responses.

Get spark application

Get status of spark application

GET /spark_engines/{engine_id}/applications/{application_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_engine_application.get

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • application_id
    Required*
    string

    Application id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/applications/{application_id}"

Response

Response Body

SparkEngineApplicationStatus

Engine Application Status

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "application_details": {
    "application": "/opt/ibm/spark/examples/src/main/python/wordcount.py",
    "conf": {
      "ae.spark.executor.count": "1",
      "spark.app.name": "MyJob",
      "spark.driver.cores": "1",
      "spark.driver.memory": "4G",
      "spark.eventLog.enabled": "true",
      "spark.executor.cores": "1",
      "spark.executor.memory": "4G"
    },
    "env": {
      "SAMPLE_ENV_KEY": "SAMPLE_VALUE"
    },
    "spark_version": "3.3"
  },
  "application_id": "ff053468-9d89-468e-a5af-bd96b88c5bf5",
  "auto_termination_time": "2024-05-16T06:56:51.469Z",
  "end_time": "2024-04-16T06:57:19.912Z",
  "finish_time": "2024-04-16T06:57:19.912Z",
  "id": "ff053468-9d89-468e-a5af-bd96b88c5bf5",
  "return_code": "0",
  "spark_application_id": "app-20240416065651-0000",
  "spark_application_name": "PythonWordCount",
  "spark_version": "3.3",
  "start_time": "2024-04-16T06:56:51.469Z",
  "state": "finished",
  "submission_time": "2024-04-16T06:56:22.909Z"
}

Get spark engine catalogs

Get list of all catalogs attached to a spark engine

GET /spark_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_engine_catalog.list

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/catalogs' -H 'accept: application/json' -H "AuthInstanceId: {instance id}"

Response

Response Body

CatalogCollection

GetCatalogs OK

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "catalogs": [
    {
      "actions": [
        "action1",
        "action2"
      ],
      "associated_buckets": [],
      "associated_databases": [],
      "associated_engines": [],
      "catalog_name": "iceberg_data",
      "sync_exception": [],
      "tags": [
        "tag1",
        "tag2"
      ]
    },
    {
      "actions": [
        "action3",
        "action4"
      ],
      "associated_buckets": [],
      "associated_databases": [],
      "associated_engines": [],
      "catalog_name": "hudi",
      "sync_exception": [],
      "tags": [
        "tag3",
        "tag4"
      ]
    },
    {
      "actions": [
        "action5",
        "action6"
      ],
      "associated_buckets": [],
      "associated_databases": [],
      "associated_engines": [],
      "catalog_name": "delta_lake",
      "sync_exception": [],
      "tags": [
        "tag5",
        "tag6"
      ]
    }
  ]
}

Associate catalogs to spark engine

Associate one or more catalogs to a spark engine

POST /spark_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_engine_catalog.add

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Request Body

Required*
CatalogPrototype

Request body

Examples: 
{
  "catalog_names": "iceberg_data, catalog02"
}
  • curl -X 'POST' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/catalogs' -H 'accept: application/json' -d '{ "catalog_names": "catalog_name01,catalog_name02" }' -H "AuthInstanceId: {instance_id}"

Response

Response Body

Catalog

Define the catalog details

Status Code

  • 201

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "catalogs": [
    {
      "actions": [
        "action1",
        "action2"
      ],
      "catalog_name": "iceberg_data",
      "sync_exception": [],
      "tags": [
        "tag1",
        "tag2"
      ]
    }
  ]
}

Disassociate catalogs from a spark engine

Disassociate one or more catalogs from a spark engine

DELETE /spark_engines/{engine_id}/catalogs

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_engine_catalog.remove

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Query Parameters

  • catalog_names
    Required*
    string

    Catalog id(s) to be stopped, comma separated

    Possible values: 1 ≤ length ≤ 1000, Value must match regular expression ^[a-zA-Z0-9\-_,]+$

  • curl -X 'DELETE' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/catalogs?catalog_names=catalog_name1' -H 'accept: */*' -H "AuthInstanceId: {instance id}"

Response

Status Code

  • 204

    No Content

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

No Sample Response

This method does not specify any sample responses.

Get spark engine catalog

Get catalog attached to spark engine

GET /spark_engines/{engine_id}/catalogs/{catalog_id}

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_engine_catalog.get

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • catalog_id
    Required*
    string

    catalog id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-_]+$

  • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/catalogs/{catalog_id}' -H 'accept: application/json' -H "AuthInstanceId: {instance id}"

Response

Response Body

Catalog

Define the catalog details

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "actions": [],
  "associated_buckets": [
    "d501dc2b-05db-4d93-9aed-edf93c7bb0f6-customer"
  ],
  "associated_databases": [],
  "associated_engines": [
    "presto459",
    "spark422"
  ],
  "catalog_name": "iceberg_data",
  "catalog_type": "iceberg",
  "created_by": "user@example.com",
  "created_on": "1713155437",
  "description": "icebarg_data",
  "hostname": "hostname",
  "port": "<port>",
  "sync_exception": [],
  "tags": [],
  "thrift_uri": "thrift://<hostname>:<port>"
}

Get spark history server

Get spark history server

GET /spark_engines/{engine_id}/history_server

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_engine.history_server

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/history_server"

Response

Response Body

SparkHistoryServer

Native spark history server

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "auto_termination_time": "2022-02-24T07:37:47Z",
  "cores": "1",
  "memory": "4G",
  "start_time": "2022-02-21T07:37:47Z",
  "state": "started"
}

Start spark history server

Start spark history server

POST /spark_engines/{engine_id}/history_server

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_engine.history_server

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

Request Body

StartSparkEngineHistoryServerParamsBody

Start history server parameter

  • curl -X POST -H "content-type: application/json" -H "AuthInstanceId: {instance id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/history_server"

Response

Response Body

SparkHistoryServer

Native spark history server

Status Code

  • 201

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "auto_termination_time": "2022-02-24T07:37:47Z",
  "cores": "1",
  "memory": "4G",
  "start_time": "2022-02-21T07:37:47Z",
  "state": "started"
}

Stop spark history server

Stop spark history server

DELETE /spark_engines/{engine_id}/history_server

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_engine.history_server

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Path Parameters

  • engine_id
    Required*
    string

    engine id

    Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • curl -X DELETE -H "content-type: application/json" -H "AuthInstanceId: {instance id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/spark_engines/{engine_id}/history_server"

Response

Status Code

  • 204

    No Content

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

No Sample Response

This method does not specify any sample responses.

List spark version

List spark version

GET /spark_versions

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.spark_versions.read

Request

Custom Headers

  • AuthInstanceId
    string

    Instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Response

Response Body

ListSparkVersionsOKBody

List spark version

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "response": {
    "message": "list all spark versions",
    "message_code": "Success"
  },
  "spark_versions": {
    "gluten": [
      {
        "display_name": "3.4",
        "value": "3.4"
      }
    ],
    "spark": [
      {
        "display_name": "3.4",
        "value": "3.4"
      },
      {
        "display_name": "3.5",
        "value": "3.5"
      }
    ]
  }
}

Get all instance details

Get all engine details

GET /instance_details

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.instance_detail.list

Request

Custom Headers

  • AuthInstanceId
    string

    CRN

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

  • curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/instance_details"

Response

Response Body

WatsonxInstanceDetailsCollection

List all Instance Details

Status Code

  • 200

    Success

  • 400

    Error

  • 401

    Unauthorized

  • 404

    Not found

  • 500

    Internal server error

Example responses
  • {
  "engines_services": [
    {
      "details": [
        {
          "SSL_certificate": "presto-java1-ssl-certificate",
          "external": {
            "hostname": "presto-java1-external-hostname",
            "port": 443
          },
          "id": "presto-java1-id",
          "internal": {
            "hostname": "presto-java1-internal-hostname",
            "port": 8443
          },
          "name": "PrestoJava1",
          "version": "Java"
        },
        {
          "SSL_certificate": "presto-java2-ssl-certificate",
          "external": {
            "hostname": "presto-java2-external-hostname",
            "port": 443
          },
          "id": "presto-java2-id",
          "internal": {
            "hostname": "presto-java2-internal-hostname",
            "port": 8443
          },
          "name": "PrestoJava2",
          "version": "Java"
        },
        {
          "SSL_certificate": "presto-cpp1-ssl-certificate",
          "external": {
            "hostname": "presto-cpp1-external-hostname",
            "port": 443
          },
          "id": "presto-cpp1-id",
          "internal": {
            "hostname": "presto-cpp1-internal-hostname",
            "port": 8443
          },
          "name": "PrestoCpp1",
          "version": "C++"
        },
        {
          "SSL_certificate": "presto-cpp2-ssl-certificate",
          "external": {
            "hostname": "presto-cpp2-external-hostname",
            "port": 443
          },
          "id": "presto-cpp2-id",
          "internal": {
            "hostname": "presto-cpp2-internal-hostname",
            "port": 8443
          },
          "name": "PrestoCpp2",
          "version": "C++"
        }
      ],
      "type": "Presto"
    },
    {
      "details": [
        {
          "SSL_certificate": "spark1-ssl-certificate",
          "default_configs": {
            "config1": "value1",
            "config2": "value2"
          },
          "id": "spark1-id",
          "name": "Spark1",
          "port": 443,
          "spark_engine_endpoint": "spark1-engine-endpoint",
          "watsonx_data_application_endpoint": "spark1-watsonx-data-app-endpoint"
        },
        {
          "SSL_certificate": "spark2-ssl-certificate",
          "default_configs": {
            "config1": "value1",
            "config2": "value2"
          },
          "id": "spark2-id",
          "name": "Spark2",
          "port": 443,
          "spark_engine_endpoint": "spark2-engine-endpoint",
          "watsonx_data_application_endpoint": "spark2-watsonx-data-app-endpoint"
        }
      ],
      "type": "Spark"
    },
    {
      "details": [
        {
          "SSL_certificate": "milvus1-ssl-certificate",
          "grpc-api-endpoint": {
            "hostname": "milvus1-grpc-hostname",
            "port": 19530
          },
          "id": "milvus1-id",
          "name": "Milvus1",
          "rest-api-endpoint": {
            "hostname": "milvus1-rest-hostname",
            "port": 443
          }
        },
        {
          "SSL_certificate": "milvus2-ssl-certificate",
          "grpc-api-endpoint": {
            "hostname": "milvus2-grpc-hostname",
            "port": 19530
          },
          "id": "milvus2-id",
          "name": "Milvus2",
          "rest-api-endpoint": {
            "hostname": "milvus2-rest-hostname",
            "port": 443
          }
        }
      ],
      "type": "Milvus"
    },
    {
      "details": [
        {
          "SSL_certificate": "cas-ssl-certificate",
          "hostname": "cas-hostname",
          "id": "cas-id",
          "port": 443
        }
      ],
      "type": "Content_Awareness_Storage"
    },
    {
      "details": [
        {
          "SSL_certificate": "cpg-ssl-certificate",
          "hostname": "cpg-hostname",
          "id": "cpg-id",
          "port": 443
        }
      ],
      "type": "Common_Policy_Gateway"
    }
  ],
  "Watsonx_data_instance": {
    "hostname": "your-instance-hostname",
    "instance_CRN": "your-instance-crn",
    "instance_ID": "your-instance-id",
    "port": 1234,
    "ssl_certificate": "your-ssl-certificate"
  }
}

Get engines and services detail

Get engines and services detail

GET /instance_details/engines_services

Authorization

To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

  • lakehouse.instance__service_details.list

Request

Custom Headers

  • AuthInstanceId
    string

    watsonx.data instance ID

    Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

Query Parameters

  • target
    Required*
    string

    Target type (e.g., 'cpd', 'generic')

    Possible values: 1 ≤ length ≤ 100, Value must match regular expression ^[a-zA-Z0-9\-]+$

  • internal_host
    boolean

    Internal host

    Default: false

    Response

    Response Body

    EnginesServicesDetails

    Engine service details

    Status Code

    • 200

      Success

    • 400

      Error

    • 401

      Unauthorized

    • 404

      Not found

    • 500

      Internal server error

    No Sample Response

    This method does not specify any sample responses.

    Get services details

    Get services details for a given engine or service type.

    GET /instance_details/engines_services/{engine_or_service_type}

    Authorization

    To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

    • lakehouse.instance_detail.get

    Request

    Custom Headers

    • AuthInstanceId
      string

      watsonx.data instance ID

      Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

    Path Parameters

    • engine_or_service_type
      Required*
      string

      Type of engine or service (e.g., 'milvus', 'presto')

      Possible values: 1 ≤ length ≤ 100, Value must match regular expression ^[a-zA-Z0-9\-]+$

    Query Parameters

    • target
      Required*
      string

      Target type (e.g., 'cpd', 'generic')

      Possible values: 1 ≤ length ≤ 100, Value must match regular expression ^[a-zA-Z0-9\-]+$

    • internal_host
      boolean

      Internal host

      Default: false

      Response

      Response Body

      ServicesDetails

      Details of engines or services for a specific target type.

      Status Code

      • 200

        Success

      • 400

        Bad Request

      • 401

        Unauthorized

      • 404

        Not Found

      • 500

        Internal Server Error

      No Sample Response

      This method does not specify any sample responses.

      Get Milvus service detail

      Get Milvus service detail

      GET /instance_details/engines_services/{engine_or_service_type}/id/{id}

      Authorization

      To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

      • lakehouse.instance_detail.get

      Request

      Custom Headers

      • AuthInstanceId
        string

        CRN

        Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

      Path Parameters

      • engine_or_service_type
        Required*
        string

        Type of engine or service (e.g., 'milvus', 'presto')

        Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

      • id
        Required*
        string

        Service ID

        Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

      Query Parameters

      • target
        Required*
        string

        Target type (e.g., 'cpd', 'generic')

        Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

      • database
        string

        Database name in milvus

        Possible values: 1 ≤ length ≤ 1000, Value must match regular expression ^[a-zA-Z0-9\-]+$

      • internal_host
        boolean

        Internal host

        Default: false

        Response

        Response Body

        ConnectionPropertiesDetails

        Get Milvus Service Detail for target type = cpd

        Status Code

        • 200

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 403

          Forbidden

        • 404

          Not found

        • 500

          Internal server error

        No Sample Response

        This method does not specify any sample responses.

        List all registered catalogs

        List all registered catalogs

        GET /catalogs

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.catalog.list

        Request

        Custom Headers

        • AuthInstanceId
          string

          Instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

        Response

        Response Body

        CatalogCollection

        GetCatalogs OK

        Status Code

        • 200

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        Example responses
        • {
  "catalogs": []
}

        Get catalog properties by catalog_id

        Get catalog properties of a catalog identified by catalog_id

        GET /catalogs/{catalog_id}

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.catalog.get

        Request

        Custom Headers

        • AuthInstanceId
          string

          Instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Path Parameters

        • catalog_id
          Required*
          string

          catalog ID

          Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-_]+$

        • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

        Response

        Response Body

        Catalog

        Define the catalog details

        Status Code

        • 200

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        Example responses
        • {
  "actions": [
    "view",
    "update",
    "delete"
  ],
  "associated_buckets": [
    "ibm_cos_bucket"
  ],
  "associated_databases": [
    "iceberg_data"
  ],
  "associated_engines": [
    "presto367"
  ],
  "catalog_name": "iceberg_data",
  "catalog_type": "iceberg",
  "created_by": "user@domain.com",
  "created_on": "1700633239",
  "description": "catalog description",
  "hostname": "1234-xyz456-abc4321.lakehouse.dev.appdomain.cloud",
  "port": "31504",
  "sync_exception": [
    "table is corrupted",
    "table metadata not there"
  ],
  "tags": [
    "tag1",
    "tag2"
  ],
  "thrift_uri": "thrift://samplehost-catalog:4354"
}

        Delete catalog

        Delete catalog identified by catalog_id

        DELETE /catalogs/{catalog_id}

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.catalog.delete

        Request

        Custom Headers

        • AuthInstanceId
          string

          watsonx.data instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Path Parameters

        • catalog_id
          Required*
          string

          catalog ID

          Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-_]+$

        Query Parameters

        • skip_mds_call
          boolean

          Skip MDS call when Unity catalog is added

          Default: false

        • curl -X 'DELETE' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}' -H 'accept: application/json' -H 'AuthInstanceId: {Instance ID}'

        Response

        Status Code

        • 204

          No Content

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        No Sample Response

        This method does not specify any sample responses.

        List all schemas

        List all schemas in catalog

        GET /catalogs/{catalog_id}/schemas

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.schema.list

        Request

        Custom Headers

        • AuthInstanceId
          string

          Instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Path Parameters

        • catalog_id
          Required*
          string

          Catalog name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+$

        Query Parameters

        • engine_id
          Required*
          string

          Engine name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

        Response

        Response Body

        ListSchemasOKBody

        GetSchemas OK

        Status Code

        • 200

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        Example responses
        • {
  "response": {
    "message": "create schema",
    "message_code": "success"
  },
  "schemas": [
    "schema1",
    "schema2"
  ]
}

        Create schema

        Create a new schema

        POST /catalogs/{catalog_id}/schemas

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.schema.create

        Request

        Custom Headers

        • AuthInstanceId
          string

          Instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Path Parameters

        • catalog_id
          Required*
          string

          Catalog name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+$

        Query Parameters

        • engine_id
          Required*
          string

          Engine name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        Request Body

        Required*
        CreateSchemaBody

        Request body

        • curl -X 'POST' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}' -H 'Content-Type: application/json' -d '{
  "bucket_name": "sample-bucket", 
  "custom_path": "sample-path", 
  "schema_name": "SampleSchema1" 
}'

        Response

        Response Body

        CreateSchemaCreatedBody

        success response

        Status Code

        • 201

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        Example responses
        • {
  "response": {
    "message": "create schema",
    "message_code": "success"
  }
}

        Delete schema

        Delete a schema

        DELETE /catalogs/{catalog_id}/schemas/{schema_id}

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.schema.delete

        Request

        Custom Headers

        • AuthInstanceId
          string

          Instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Path Parameters

        • catalog_id
          Required*
          string

          Catalog name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+$

        • schema_id
          Required*
          string

          URL encoded Schema name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^.*$

        Query Parameters

        • engine_id
          Required*
          string

          Engine name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        • curl -X 'DELETE' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}?engine_id={engine_id}' -H 'accept: */*' -H 'AuthInstanceId: {instance_id}'

        Response

        Status Code

        • 204

          No Content

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        No Sample Response

        This method does not specify any sample responses.

        List all tables

        List all tables in a schema in a catalog for a given engine

        GET /catalogs/{catalog_id}/schemas/{schema_id}/tables

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.table.list

        Request

        Custom Headers

        • AuthInstanceId
          string

          Instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Path Parameters

        • catalog_id
          Required*
          string

          catalog id

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        • schema_id
          Required*
          string

          URL encoded schema name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+$

        Query Parameters

        • engine_id
          Required*
          string

          engine id

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

        Response

        Response Body

        TableCollection

        tables list

        Status Code

        • 200

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        Example responses
        • {
  "tables": [
    "customer_data",
    "user_data"
  ]
}

        Get table details

        Get details of a given table in a catalog and schema

        GET /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.table.get

        Request

        Custom Headers

        • AuthInstanceId
          string

          Instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Path Parameters

        • catalog_id
          Required*
          string

          catalog id

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+$

        • schema_id
          Required*
          string

          URL encoded schema name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^.*$

        • table_id
          Required*
          string

          URL encoded table name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^.*$

        Query Parameters

        • engine_id
          Required*
          string

          engine id

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

        Response

        Response Body

        Table

        GetColumns OK

        Status Code

        • 200

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        Example responses
        • {
  "columns": [
    {
      "column_name": "expenses",
      "comment": "",
      "extra": "",
      "precision": "1234567890",
      "type": "varchar"
    },
    {
      "column_name": "amount",
      "comment": "",
      "extra": "",
      "scale": "10",
      "precision": "1234567890",
      "type": "integer"
    }
  ],
  "table_name": "newtable"
}

        Delete table

        Delete table for a given schema and catalog

        DELETE /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.table.delete

        Request

        Custom Headers

        • AuthInstanceId
          string

          Instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Path Parameters

        • catalog_id
          Required*
          string

          catalog id

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+$

        • schema_id
          Required*
          string

          URL encoded schema name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^.*$

        • table_id
          Required*
          string

          URL encoded table name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^.*$

        Query Parameters

        • engine_id
          Required*
          string

          engine id

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        • curl -X 'DELETE' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}?engine_id={engine_id}' -H 'accept: */*' -H 'AuthInstanceId: {instance_id}'

        Response

        Status Code

        • 204

          No Content

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        No Sample Response

        This method does not specify any sample responses.

        Rename table

        Rename table

        PATCH /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.table.update

        Request

        Custom Headers

        • AuthInstanceId
          string

          Instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Path Parameters

        • catalog_id
          Required*
          string

          catalog id

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+$

        • schema_id
          Required*
          string

          URL encoded schema name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^.*$

        • table_id
          Required*
          string

          URL encoded table name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^.*$

        Query Parameters

        • engine_id
          Required*
          string

          engine id

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        Request Body

        Required*
        application/merge-patch+jsonTablePatch

        Request body

        • curl -X 'PATCH' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}' -H 'Content-Type: application/merge-patch+json' -d '{ 
  "table_name": "updated_table_name", 
}'

        Response

        Response Body

        Table

        GetColumns OK

        Status Code

        • 200

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        Example responses
        • {
  "columns": [
    {
      "column_name": "expenses",
      "comment": "",
      "extra": "",
      "precision": "1234567890",
      "type": "varchar"
    },
    {
      "column_name": "amount",
      "comment": "",
      "extra": "",
      "scale": "10",
      "precision": "1234567890",
      "type": "integer"
    }
  ],
  "table_name": "newtable"
}

        List all columns of a table

        List all columns of a table in a given a schema for a given catalog

        GET /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/columns

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.column.list

        Request

        Custom Headers

        • AuthInstanceId
          string

          Instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Path Parameters

        • catalog_id
          Required*
          string

          catalog id

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        • schema_id
          Required*
          string

          URL encoded schema name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+$

        • table_id
          Required*
          string

          URL encoded schema name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+$

        Query Parameters

        • engine_id
          Required*
          string

          engine id

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/columns?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

        Response

        Response Body

        ColumnCollection

        list of columns in a table

        Status Code

        • 200

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        Example responses
        • {
  "columns": [
    {
      "column_name": "expenses",
      "comment": "expenses",
      "extra": "",
      "precision": "1234567890",
      "type": "varchar"
    },
    {
      "column_name": "amount",
      "comment": "amount",
      "extra": "",
      "scale": "10",
      "type": "integer"
    }
  ]
}

        Add column(s)

        Add one or multiple columns to a table in a schema for a given catalog

        POST /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/columns

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.column.create

        Request

        Custom Headers

        • AuthInstanceId
          string

          Instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Path Parameters

        • catalog_id
          Required*
          string

          catalog id

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        • schema_id
          Required*
          string

          URL encoded schema name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+$

        • table_id
          Required*
          string

          URL encoded schema name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+$

        Query Parameters

        • engine_id
          Required*
          string

          engine id

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        Request Body

        Required*
        ColumnPrototype

        Request body

        Examples: 
        {
  "columns": [
    {
      "column_name": "expenses",
      "type": "varchar"
    }
  ]
}
        • curl -X 'POST' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/columns?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}' -H 'Content-Type: application/json' -d '{
  "bucket_name": "sample-bucket", 
  "custom_path": "sample-path", 
  "schema_name": "SampleSchema1" 
}'

        Response

        Response Body

        ColumnCollection

        list of columns in a table

        Status Code

        • 201

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        Example responses
        • {
  "columns": [
    {
      "column_name": "expenses",
      "comment": "expenses",
      "extra": "",
      "precision": "1234567890",
      "type": "varchar"
    },
    {
      "column_name": "amount",
      "comment": "amount",
      "extra": "",
      "scale": "10",
      "type": "integer"
    }
  ]
}

        Delete column

        Delete column in a table for a given schema and catalog

        DELETE /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/columns/{column_id}

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.column.delete

        Request

        Custom Headers

        • AuthInstanceId
          string

          Instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Path Parameters

        • catalog_id
          Required*
          string

          catalog id

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        • schema_id
          Required*
          string

          URL encoded schema name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+$

        • table_id
          Required*
          string

          URL encoded schema name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+$

        • column_id
          Required*
          string

          URL encoded schema name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+$

        Query Parameters

        • engine_id
          Required*
          string

          engine id

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        • curl -X 'DELETE' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}?engine_id={engine_id}' -H 'accept: */*' -H 'AuthInstanceId: {instance_id}'

        Response

        Status Code

        • 204

          No Content

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        No Sample Response

        This method does not specify any sample responses.

        Alter column

        Update the given column - rename column

        PATCH /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/columns/{column_id}

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.column.update

        Request

        Custom Headers

        • AuthInstanceId
          string

          Instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Path Parameters

        • catalog_id
          Required*
          string

          catalog id

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        • schema_id
          Required*
          string

          URL encoded schema name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+$

        • table_id
          Required*
          string

          URL encoded schema name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+$

        • column_id
          Required*
          string

          URL encoded schema name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+$

        Query Parameters

        • engine_id
          Required*
          string

          engine id

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        Request Body

        Required*
        application/merge-patch+jsonColumnPatch

        Request body

        Examples: 
        {
  "column_name": "expenses"
}
        • curl -X 'PATCH' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}' -H 'Content-Type: application/merge-patch+json' -d '{ 
  "column_name": "updated_column_name", 
}'

        Response

        Response Body

        Column

        Column

        Status Code

        • 200

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        Example responses
        • {
  "column_name": "expenses,",
  "comment": "expenses column",
  "extra": "varchar,",
  "precision": "1234567890",
  "type": "varchar,"
}

        Get table snapshots

        List all table snapshots

        GET /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/snapshots

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.table_snapshot.list

        Request

        Custom Headers

        • AuthInstanceId
          string

          Instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Path Parameters

        • catalog_id
          Required*
          string

          Catalog ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+$

        • schema_id
          Required*
          string

          Schema ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^.*$

        • table_id
          Required*
          string

          Table ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^.*$

        Query Parameters

        • engine_id
          Required*
          string

          Engine name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/snapshots?engine_id={engine_id}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

        Response

        Response Body

        TableSnapshotCollection

        TableSnapshot OK

        Status Code

        • 200

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        Example responses
        • {
  "snapshots": [
    {
      "committed_at": "1609379392",
      "operation": "alter",
      "snapshot_id": "2332342122211222",
      "summary": ""
    }
  ]
}

        Rollback table to snapshot

        Rollback table to a snapshot

        POST /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/rollback

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.dashboard.view

        Request

        Custom Headers

        • AuthInstanceId
          string

          CRN

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Path Parameters

        • catalog_id
          Required*
          string

          Catalog ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        • schema_id
          Required*
          string

          Schema ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^.*$

        • table_id
          Required*
          string

          Table ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^.*$

        Query Parameters

        • engine_id
          Required*
          string

          Engine name

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-]+$

        Request Body

        Required*
        Snapshot

        Request body

        • curl -X 'POST' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/rollback?engine_id={engine_id}' -H 'accept: application/json' -H "Content-Type: application/json" -d '{ "snapshot_id": "snapshot_id", }' -H 'AuthInstanceId: {instance_id}'

        Response

        Response Body

        ReplaceSnapshotCreatedBody

        success response

        Status Code

        • 201

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        Example responses
        • {
  "response": {
    "message": "Rollback to a table snapshot",
    "message_code": "success"
  }
}

        External Iceberg table registration

        Synchronize the external Iceberg table registration for a catalog identified by catalog_id.

        PATCH /catalogs/{catalog_id}/sync

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.catalog_sync.update

        Request

        Custom Headers

        • AuthInstanceId
          string

          Instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Path Parameters

        • catalog_id
          Required*
          string

          catalog ID

          Possible values: 1 ≤ length ≤ 1000, Value must match regular expression ^[a-zA-Z0-9\-_]+$

        Request Body

        Required*
        application/merge-patch+jsonSyncCatalogs

        Request body

        • curl -X 'PATCH' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/sync' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}' -H 'Content-Type: application/merge-patch+json' -d '{ 
  "auto_add_new_tables": true, 
  "sync_iceberg_md": true 
}'

        Response

        Response Body

        UpdateSyncCatalogOKBody

        success response

        Status Code

        • 200

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        Example responses
        • {
  "response": {
    "message": "sync catalog",
    "message_code": "success"
  }
}

        Get SAL Integrations

        Get SAL Integration

        GET /sal_integrations

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.sal_integration.get

        Request

        Custom Headers

        • AuthInstanceId
          string

          watsonx.data instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations"

        Response

        Response Body

        SalIntegration

        Sal Integration object

        Status Code

        • 200

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        Example responses
        • {
  "category_id": "10e64285-bf37-4d5d-b759-bc6a46589234",
  "engine_id": "presto-01",
  "governance_scope_id": "10e64285-bf37-4d5d-b759-bc6a46589234",
  "governance_scope_type": "category",
  "instance_id": "18b49d7a-9519-4539-8db5-ff080623c226",
  "status": "active",
  "storage_resource_crn": "crn:v1:staging:public:cloud-object-storage:global:a/04e9bc47612523rr19ac22759cb69bebd:asd23df-6ada-45ff-bfe8-4343222",
  "storage_type": "bmcos_object_storage",
  "timestamp": "1715056266",
  "trial_plan": false,
  "username": "xyz@abc.com"
}

        Create sal integration with wxd

        Add or create a new sal integration

        POST /sal_integrations

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.sal_integration.create

        Request

        Custom Headers

        • AuthInstanceId
          string

          watsonx.data instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Request Body

        Required*
        SalIntegrationPrototype

        Request body

        • curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/bucket_registrations" -d '{ 
"bucket_display_name": "sample_bucket", 
"bucket_type": "ibm_cos", 
"description": "", 
"managed_by": "customer", 
"region": "us-south", 
"bucket_details": { 
  "bucket_name": "hive-test-cos", 
  "endpoint": "https://s3.us-west.cloud-object-storage.test.appdomain.cloud", 
  "access_key": "cfgr14cfgtffegfdew34556", 
  "secret_key": "12345dsadcvfbddsadfgfvcxasdf" 
}, 
"associated_catalog": { 
  "catalog_name": "sample_catalog", 
  "catalog_type": "iceberg" 
} 
}'

        Response

        Response Body

        SalIntegration

        Sal Integration object

        Status Code

        • 201

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        Example responses
        • {
  "category_id": "10e64285-bf37-4d5d-b759-bc6a46589234",
  "engine_id": "presto-01",
  "governance_scope_id": "10e64285-bf37-4d5d-b759-bc6a46589234",
  "governance_scope_type": "category",
  "instance_id": "18b49d7a-9519-4539-8db5-ff080623c226",
  "status": "active",
  "storage_resource_crn": "crn:v1:staging:public:cloud-object-storage:global:a/04e9bc47612523rr19ac22759cb69bebd:asd23df-6ada-45ff-bfe8-4343222",
  "storage_type": "bmcos_object_storage",
  "timestamp": "1715056266",
  "trial_plan": false,
  "username": "xyz@abc.com"
}

        Delete sal-wxd integration

        Delete a sal-wxd integration

        DELETE /sal_integrations

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.sal_integration.delete

        Request

        No Request Parameters

        This method does not accept any request parameters.

        • curl -X DELETE -H "accept: application/json" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations"

        Response

        Status Code

        • 204

          No Content

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        No Sample Response

        This method does not specify any sample responses.

        Update sal-wxd integration

        Update sal-wxd integration details

        PATCH /sal_integrations

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.sal_integration.update

        Request

        Custom Headers

        • AuthInstanceId
          string

          watsonx.data instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Request Body

        Required*
        application/merge-patch+jsonSalIntegrationPatch

        Request body

        • curl -X PATCH -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations" -d '{ 
"apikey": "{apikey}" 
}'

        Response

        Response Body

        SalIntegration

        Sal Integration object

        Status Code

        • 200

          Success

        • 400

          Error

        • 401

          Unauthorized

        • 404

          Not found

        • 500

          Internal server error

        Example responses
        • {
  "category_id": "10e64285-bf37-4d5d-b759-bc6a46589234",
  "engine_id": "presto-01",
  "governance_scope_id": "10e64285-bf37-4d5d-b759-bc6a46589234",
  "governance_scope_type": "category",
  "instance_id": "18b49d7a-9519-4539-8db5-ff080623c226",
  "status": "active",
  "storage_resource_crn": "crn:v1:staging:public:cloud-object-storage:global:a/04e9bc47612523rr19ac22759cb69bebd:asd23df-6ada-45ff-bfe8-4343222",
  "storage_type": "bmcos_object_storage",
  "timestamp": "1715056266",
  "trial_plan": false,
  "username": "xyz@abc.com"
}

        Trigger enrichment jobs on schemas and tables

        Trigger enrichment jobs on schemas and tables

        POST /sal_integrations/enrichment

        Authorization

        To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

        • lakehouse.sal_integration_enrichment.create

        Request

        Custom Headers

        • AuthInstanceId
          string

          watsonx.data instance ID

          Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

        Request Body

        Required*
        SalIntegrationEnrichmentPrototype

        Request body

        Examples: 
        {
  "enrichment_prototype": {
    "catalog": "iceberg_data",
    "operation": "create",
    "schema": "schema1",
    "tables": [
      "table1",
      "table2",
      "table3"
    ]
  }
}
        • curl -X POST -H "accept: application/json" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations/enrichment" -d '{ 
  "changes": [{ 
       "catalog": "iceberg_data",
       "operation": "create",
       "schema": "schema1",
       "tables": [
          "string"
         ]
    }]
}'

          Response

          Status Code

          • 204

            No Content

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          No Sample Response

          This method does not specify any sample responses.

          Get semantic enrichment assets associated with the schema

          Get semantic enrichment job runs associated with the schema

          GET /sal_integrations/enrichment_assets

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.sal_integration_enrichment_assets.get

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Query Parameters

          • project_id
            string

            enrichment project id

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations/enrichment_assets"

          Response

          Response Body

          SalIntegrationEnrichmentAssets

          sal integration enrichment assets

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "enrichment_asset": {
    "asset_attributes": [
      "attribute1",
      "attribute2",
      "attribute3"
    ],
    "asset_id": "ee0383b9-dcab-4c1a-b03d-bf521837b6ed",
    "asset_name": "newtable",
    "resource_key": "0000:0000:0000:0000:0000:FFFF:9EB0:04E2|31134|:/iceberg_data/new_schema/sampletable",
    "schema_name": "sampleschema"
  }
}

          Get semantic enrichment data asset associated with the table

          Get semantic enrichment data asset associated with the table

          GET /sal_integrations/enrichment_data_asset

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.sal_integration_enrichment_assets.get

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Query Parameters

          • project_id
            string

            enrichment project id

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          • asset_id
            string

            enrichment data asset id

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations/enrichment_data_asset"

          Response

          Response Body

          SalIntegrationEnrichmentDataAsset

          semantic enrichment data asset

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "asset": "{}"
}

          Get semantic enrichment job run logs associated with the job run

          Get semantic enrichment job run logs associated with the job run

          GET /sal_integrations/enrichment_job_run_logs

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.sal_integration_enrichment_job_run_logs.get

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Query Parameters

          • job_id
            string

            enrichment job id

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          • job_run_id
            string

            enrichment job run id

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          • project_id
            string

            enrichment project id

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations/enrichment_job_run_logs"

          Response

          Response Body

          SalIntegrationEnrichmentJobRunLogs

          semantic enrichment job run logs

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "results": [
    "example1",
    "example2",
    "example3"
  ],
  "total_count": 12
}

          Get semantic enrichment job runs associated with the schema

          Get semantic enrichment job runs associated with the schema

          GET /sal_integrations/enrichment_job_runs

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.sal_integration_enrichment_job_runs.get

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Query Parameters

          • job_id
            string

            enrichment job id

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          • project_id
            string

            enrichment project id

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations/enrichment_job_runs"

          Response

          Response Body

          SalIntegrationEnrichmentJobRun

          semantic enrichment job run

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "response": "{}"
}

          Get semantic enrichment jobs associated with the schema

          Get semantic enrichment jobs associated with the schema

          GET /sal_integrations/enrichment_jobs

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.sal_integration_enrichment_jobs.get

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Query Parameters

          • wkc_project_id
            string

            ikc project id

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations/enrichment_jobs"

          Response

          Response Body

          SalIntegrationEnrichmentJobs

          Sal Integration Mappings object

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "jobs": {
    "results": [
      {
        "entity": {
          "job": {
            "asset_ref": "8688a3b6-a946-499e-a93c-b7d099db80dd",
            "asset_ref_type": "metadata_enrichment_area",
            "configuration": {
              "env_type": "",
              "env_variables": []
            },
            "enable_notifications": false,
            "future_scheduled_runs": [],
            "last_run_initiator": "deprecated field",
            "last_run_status": "deprecated field",
            "last_run_status_timestamp": 0,
            "last_run_time": "deprecated field",
            "project_name": "SAL Mapping /iceberg_data/new_schema 9aae5be3-87cf-4c31-b17d-9256ab42c14e",
            "schedule_creator_id": "",
            "schedule_id": "",
            "schedule_info": {},
            "task_credentials_support": {
              "account_id": "04e9bc4761254b719ac22759cb69bebd",
              "task_credentials_enabled": true,
              "user_id": "IBMid-55000832RK"
            }
          }
        },
        "metadata": {
          "asset_id": "ea73ce44-8aa0-4c75-bd69-6ca7074a1030",
          "name": "SAL_MDE job",
          "owner_id": "IBMid-55000832RK",
          "version": 0
        }
      }
    ],
    "total_rows": 1
  }
}

          Get list of uploaded glossary terms

          Get list of uploaded glossary terms

          GET /sal_integrations/glossary_terms

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.sal_integration_glossary_terms.get

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations/glossary_terms"

          Response

          Response Body

          SalIntegrationGlossaryTerms

          Sal integration glossary terms

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "glossary_term": {
    "description": "First Name",
    "name": "Name"
  }
}

          Get wkc catalog and project mapped to the schema

          Get wkc catalog and project mapped to the schema

          GET /sal_integrations/mappings

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.sal_integration_mappings.get

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Query Parameters

          • catalog_name
            Required*
            string

            catalog name

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          • schema_name
            Required*
            string

            schema name

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations/mappings?catalog_name={catalog_name}&schema_name={schema_name}"

          Response

          Response Body

          SalIntegrationMappings

          Sal Integration Mappings object

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "wkc_catalog_id": "18b49d7a-9519-4539-8db5-ff080623c226",
  "wkc_project_id": "9519bd7a-9519-8db5-4539-ff080623c226"
}

          Get metadata enrichment global settings

          Get metadata enrichment global settings

          GET /sal_integrations/metadata_enrichment_global_settings

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.sal_integration_metadata_enrichment_global_settings.get

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations/metadata_enrichment_global_settings"

          Response

          Response Body

          SalIntegrationEnrichmentSettings

          Sal Integration Enrichment Settings objects

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "semantic_expansion": {
    "description_generation": true,
    "description_generation_configuration": {
      "assignment_threshold": 0.14,
      "suggestion_threshold": 0.9
    },
    "name_expansion": true,
    "name_expansion_configuration": {
      "assignment_threshold": 0.1,
      "suggestion_threshold": 0.1
    }
  },
  "term_assignment": {
    "class_based_assignments": false,
    "evaluate_negative_assignments": false,
    "llm_based_assignments": false,
    "ml_based_assignments_custom": false,
    "ml_based_assignments_default": false,
    "name_matching": false,
    "term_assignment_threshold": 0.3,
    "term_suggestion_threshold": 0.4
  }
}

          Add metadata enrichment global settings

          Add metadata enrichment global settings

          POST /sal_integrations/metadata_enrichment_global_settings

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.sal_integration_metadata_enrichment_global_settings.post

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Request Body

          Required*
          SalIntegrationEnrichmentSettings

          Request body

          Examples: 
          {
  "semantic_expansion": {
    "description_generation": true,
    "description_generation_configuration": {
      "assignment_threshold": 0.14,
      "suggestion_threshold": 0.9
    },
    "name_expansion": true,
    "name_expansion_configuration": {
      "assignment_threshold": 0.1,
      "suggestion_threshold": 0.1
    }
  },
  "term_assignment": {
    "class_based_assignments": false,
    "evaluate_negative_assignments": false,
    "llm_based_assignments": false,
    "ml_based_assignments_custom": false,
    "ml_based_assignments_default": false,
    "name_matching": false,
    "term_assignment_threshold": 0.3,
    "term_suggestion_threshold": 0.4
  }
}
          • curl -X POST -H "accept: application/json" -H "Content-Type: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations/metadata_enrichment_global_settings" -d '{ 
  "semantic_expansion": { 
    "description_generation": true, 
    "description_generation_configuration": {
      "assignment_threshold": 0.14,
      "suggestion_threshold": 0.9
     },
   "name_expansion": true,
   "name_expansion_configuration": {
      "assignment_threshold": 0.1,
      "suggestion_threshold": 0.1
     }
  },
 "term_assignment": {
   "class_based_assignments": false,
   "evaluate_negative_assignments": false,
   "llm_based_assignments": false,
   "ml_based_assignments_custom": false,
   "ml_based_assignments_default": false,
   "name_matching": false,
   "term_assignment_threshold": 0.3,
   "term_suggestion_threshold": 0.4
 }
}'

          Response

          Response Body

          SalIntegrationEnrichmentSettings

          Sal Integration Enrichment Settings objects

          Status Code

          • 201

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "semantic_expansion": {
    "description_generation": true,
    "description_generation_configuration": {
      "assignment_threshold": 0.14,
      "suggestion_threshold": 0.9
    },
    "name_expansion": true,
    "name_expansion_configuration": {
      "assignment_threshold": 0.1,
      "suggestion_threshold": 0.1
    }
  },
  "term_assignment": {
    "class_based_assignments": false,
    "evaluate_negative_assignments": false,
    "llm_based_assignments": false,
    "ml_based_assignments_custom": false,
    "ml_based_assignments_default": false,
    "name_matching": false,
    "term_assignment_threshold": 0.3,
    "term_suggestion_threshold": 0.4
  }
}

          get metadata enrichment settings for a project

          get metadata enrichment settings for a project

          GET /sal_integrations/metadata_enrichment_settings

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.sal_integration_metadata_enrichment_settings.get

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Query Parameters

          • project_id
            string

            wkc project id

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations/metadata_enrichment_settings"

          Response

          Response Body

          SalIntegrationEnrichmentSettings

          Sal Integration Enrichment Settings objects

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "semantic_expansion": {
    "description_generation": true,
    "description_generation_configuration": {
      "assignment_threshold": 0.14,
      "suggestion_threshold": 0.9
    },
    "name_expansion": true,
    "name_expansion_configuration": {
      "assignment_threshold": 0.1,
      "suggestion_threshold": 0.1
    }
  },
  "term_assignment": {
    "class_based_assignments": false,
    "evaluate_negative_assignments": false,
    "llm_based_assignments": false,
    "ml_based_assignments_custom": false,
    "ml_based_assignments_default": false,
    "name_matching": false,
    "term_assignment_threshold": 0.3,
    "term_suggestion_threshold": 0.4
  }
}

          Add metadata enrichment settings for a project

          Add metadata enrichment settings for a project

          POST /sal_integrations/metadata_enrichment_settings

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.sal_integration_metadata_enrichment_settings.create

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Query Parameters

          • project_id
            string

            wkc project id

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          Request Body

          Required*
          SalIntegrationEnrichmentSettings

          Request body

          Examples: 
          {
  "semantic_expansion": {
    "description_generation": true,
    "description_generation_configuration": {
      "assignment_threshold": 0.14,
      "suggestion_threshold": 0.9
    },
    "name_expansion": true,
    "name_expansion_configuration": {
      "assignment_threshold": 0.1,
      "suggestion_threshold": 0.1
    }
  },
  "term_assignment": {
    "class_based_assignments": false,
    "evaluate_negative_assignments": false,
    "llm_based_assignments": false,
    "ml_based_assignments_custom": false,
    "ml_based_assignments_default": false,
    "name_matching": false,
    "term_assignment_threshold": 0.3,
    "term_suggestion_threshold": 0.4
  }
}
          • curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations/metadata_enrichment_settings" -d '{ 
  "semantic_expansion": { 
    "description_generation": true, 
    "description_generation_configuration": {
      "assignment_threshold": 0.14,
      "suggestion_threshold": 0.9
     },
   "name_expansion": true,
   "name_expansion_configuration": {
      "assignment_threshold": 0.1,
      "suggestion_threshold": 0.1
     }
  },
 "term_assignment": {
   "class_based_assignments": false,
   "evaluate_negative_assignments": false,
   "llm_based_assignments": false,
   "ml_based_assignments_custom": false,
   "ml_based_assignments_default": false,
   "name_matching": false,
   "term_assignment_threshold": 0.3,
   "term_suggestion_threshold": 0.4
 }
}'

          Response

          Status Code

          • 204

            No Content

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          No Sample Response

          This method does not specify any sample responses.

          Upload semantic enrichment business terms glossary

          Upload semantic enrichment business terms glossary

          POST /sal_integrations/upload_glossary

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.sal_integration_upload_glossary.post

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Form Parameters

          • replace_option
            Required*
            string

            glossary upload replace option

            Allowable values: [all,specified,empty]

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+$

          • glossary_csv
            binary

            Glossary CSV file

            Possible values: 1 ≤ length ≤ 104857600

          • curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H 'Content-Type: multipart/form-data' -F 'glossary_csv=@{glossary_file_name}.csv;type=text/csv' -F 'replace_option=all' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations/upload_glossary"

          Response

          Response Body

          SalIntegrationUploadGlossary

          Sal Integration Upload Glossary

          Status Code

          • 201

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "process_id": "18b49d7a-9519-4539-8db5-ff080623c226"
}

          Get status of upload glossary job

          Get status of upload glossary job

          GET /sal_integrations/upload_glossary_status

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.sal_integration_upload_glossary_status.get

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Query Parameters

          • process_id
            string

            upload process id

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/sal_integrations/upload_glossary_status"

          Response

          Response Body

          SalIntegrationUploadGlossaryStatus

          Sal Integration Upload Glossary Status

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "response": "response"
}

          Get list of milvus services

          Get list milvus services

          GET /milvus_services

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.milvus_service.read

          Request

          Custom Headers

          • AuthInstanceId
            string

            Instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          • curl -X GET -H "content-type: application/json" -H "AuthInstanceId: {instance id}" https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/milvus_services"

          Response

          Response Body

          MilvusServiceCollection

          List milvus services

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "milvus_services": [
    {
      "actions": [
        "update",
        "delete"
      ],
      "bucket_name": "wxd-milvus",
      "created_by": "username@domain.com",
      "created_on": 1700201877,
      "description": "description",
      "grpc_host": "<formation_id>.<kubernetes_cluster>.databases.appdomain.cloud",
      "grpc_port": 443,
      "https_host": "<formation_id>.<kubernetes_cluster>.databases.appdomain.cloud",
      "https_port": 443,
      "origin": "native",
      "service_display_name": "test-milvus",
      "service_id": "milvus76",
      "status": "running",
      "status_code": 0,
      "tags": [],
      "tshirt_size": "Starter",
      "type": "milvus"
    }
  ]
}

          Create milvus service

          Create milvus service

          POST /milvus_services

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.milvus_service.create

          Request

          Custom Headers

          • AuthInstanceId
            string

            Instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Request Body

          Required*
          MilvusServicePrototype

          Create milvus service

          • curl -X 'POST' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/milvus_services' -H 'accept: application/json' -H "AuthInstanceId: {instance id}" -H 'Content-Type: application/json' -d '{  "bucket_name": "Sample bucket name",  "bucket_type": "Sample bucket type",  "description": "milvus service for running sql queries",  "origin": "native",  "root_path": "Sample path",  "service_display_name": "sampleService",  "tags": [    "tag1",    "tag2"  ],  "tshirt_size": "small",   }'

          Response

          Response Body

          MilvusService

          milvus service details

          Status Code

          • 201

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "actions": [
    "update",
    "delete"
  ],
  "bucket_name": "wxd-milvus",
  "created_by": "username@domain.com",
  "created_on": 1700201877,
  "description": "description",
  "grpc_host": "<formation_id>.<kubernetes_cluster>.databases.appdomain.cloud",
  "grpc_port": 443,
  "https_host": "<formation_id>.<kubernetes_cluster>.databases.appdomain.cloud",
  "https_port": 443,
  "origin": "native",
  "service_display_name": "test-milvus",
  "service_id": "milvus76",
  "status": "running",
  "status_code": 0,
  "tags": [],
  "tshirt_size": "Starter",
  "type": "milvus"
}

          Get milvus service

          Get milvus service

          GET /milvus_services/{service_id}

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.milvus_service.get

          Request

          Custom Headers

          • AuthInstanceId
            string

            Instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Path Parameters

          • service_id
            Required*
            string

            service id

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/milvus_services/{service_id}' -H 'accept: application/json' -H "AuthInstanceId: {instance id}"

          Response

          Response Body

          MilvusService

          milvus service details

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "actions": [
    "update",
    "delete"
  ],
  "bucket_name": "wxd-milvus",
  "created_by": "username@domain.com",
  "created_on": 1700201877,
  "description": "description",
  "grpc_host": "<formation_id>.<kubernetes_cluster>.databases.appdomain.cloud",
  "grpc_port": 443,
  "https_host": "<formation_id>.<kubernetes_cluster>.databases.appdomain.cloud",
  "https_port": 443,
  "origin": "native",
  "service_display_name": "test-milvus",
  "service_id": "milvus76",
  "status": "running",
  "status_code": 0,
  "tags": [],
  "tshirt_size": "Starter",
  "type": "milvus"
}

          Delete milvus service

          Delete milvus service

          DELETE /milvus_services/{service_id}

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.milvus_service.delete

          Request

          Custom Headers

          • AuthInstanceId
            string

            Instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Path Parameters

          • service_id
            Required*
            string

            service id

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          • curl -X 'DELETE' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/milvus_services/{service_id}' -H 'accept: application/json' -H "AuthInstanceId: {instance id}"

          Response

          Status Code

          • 204

            No Content

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          No Sample Response

          This method does not specify any sample responses.

          Update milvus service

          Update details of milvus service

          PATCH /milvus_services/{service_id}

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.milvus_service.update

          Request

          Custom Headers

          • AuthInstanceId
            string

            Instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Path Parameters

          • service_id
            Required*
            string

            service id

            Possible values: 1 ≤ length ≤ 10000, Value must match regular expression ^[a-zA-Z0-9\-]+$

          Request Body

          Required*
          application/merge-patch+jsonMilvusServicePatch

          Update milvus service

          • curl -X 'PATCH' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/milvus_services/{service_id}' -H 'accept: application/json' -H 'Content-Type: application/merge-patch+json' -H "AuthInstanceId: {instance id}" -d '{  "description": "updated description for milvus service",  "engine_display_name": "sampleService",  "tags": [    "tag1",    "tag2"  ]}'

          Response

          Response Body

          MilvusService

          milvus service details

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "actions": [
    "update",
    "delete"
  ],
  "bucket_name": "wxd-milvus",
  "created_by": "username@domain.com",
  "created_on": 1700201877,
  "description": "description",
  "grpc_host": "<formation_id>.<kubernetes_cluster>.databases.appdomain.cloud",
  "grpc_port": 443,
  "https_host": "<formation_id>.<kubernetes_cluster>.databases.appdomain.cloud",
  "https_port": 443,
  "origin": "native",
  "service_display_name": "test-milvus",
  "service_id": "milvus76",
  "status": "running",
  "status_code": 0,
  "tags": [],
  "tshirt_size": "Starter",
  "type": "milvus"
}

          Get CPG and DAS endpoints

          Get Common policy gateway (CPG) and Data Access Service(DAS) endpoints

          GET /endpoints

          Request

          Custom Headers

          • AuthInstanceId
            string

            CRN

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Response

          Response Body

          EndpointCollection

          List endpoints

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "endpoints": [
    {
      "external_host": "https://cpg-svc.your-hostname.apps.your-domain.com",
      "service_type": "cpg"
    },
    {
      "external_host": "https://cas-svc.your-hostname.apps.your-domain.com",
      "service_type": "cas"
    }
  ]
}

          Get ingestion jobs

          Get list of ingestion jobs

          GET /ingestion_jobs

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.ingestion_jobs.list

          Request

          Custom Headers

          • AuthInstanceId
            Required*
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Query Parameters

          • start
            string

            Page number of requested ingestion jobs.

            Possible values: 1 ≤ length ≤ 512, Value must match regular expression ^[0-9]*$

            Default: 1

          • jobs_per_page
            int32

            Number of requested ingestion jobs.

            Possible values: 1 ≤ value ≤ 100

            Default: 1

          • curl -X GET -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/ingestion_jobs'

          Response

          Response Body

          IngestionJobCollection

          List ingestion jobs

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal Server Error

          Example responses
          • {
  "ingestion_jobs": [
    {
      "create_if_not_exist": false,
      "csv_property": {
        "encoding": "utf-8",
        "escape_character": "|",
        "field_delimiter": ",",
        "header": true,
        "line_delimiter": "\n"
      },
      "details": "Path does not exist 'demobucket/data/yellow_tripdata_2022-01.parquet'. Detail: [errno 2] No such file or directory",
      "end_timestamp": "1685088775",
      "engine_id": "spark123",
      "engine_name": "sparkdemo",
      "execute_config": {
        "driver_cores": 1,
        "driver_memory": "2G",
        "executor_cores": 1,
        "executor_memory": "2G",
        "num_executors": 1
      },
      "instance_id": "1684432229673971",
      "job_id": "ingestion-1699459946935",
      "partition_by": "col1, col2",
      "schema": "[{\"type\":\"int\",\"field_id\":0,\"header_name\":\"c1\"},{\"type\":\"string\",\"field_id\":1,\"header_name\":\"c2\"}]",
      "source_data_files": "s3://demobucket/data/yellow_tripdata_2022-01.parquet",
      "source_file_type": "csv",
      "start_timestamp": "1685084455",
      "status": "running",
      "target_table": "demodb.test.targettable",
      "username": "ibmlhadmin",
      "valdiate_csv_header": false
    },
    {
      "create_if_not_exist": false,
      "csv_property": {
        "encoding": "utf-8",
        "escape_character": "|",
        "field_delimiter": ",",
        "header": true,
        "line_delimiter": "\n"
      },
      "details": "Path does not exist 'demobucket/data/yellow_tripdata_2022-01.parquet'. Detail: [errno 2] No such file or directory",
      "end_timestamp": "1685088798",
      "engine_id": "spark123",
      "engine_name": "sparkdemo",
      "execute_config": {
        "driver_cores": 1,
        "driver_memory": "2G",
        "executor_cores": 1,
        "executor_memory": "2G",
        "num_executors": 1
      },
      "instance_id": "1684432229673972",
      "job_id": "ingestion-1699459946936",
      "partition_by": "col1, col2",
      "schema": "[{\"type\":\"int\",\"field_id\":0,\"header_name\":\"c1\"},{\"type\":\"string\",\"field_id\":1,\"header_name\":\"c2\"}]",
      "source_data_files": "s3://demobucket/data/yellow_tripdata_2022-01.parquet",
      "source_file_type": "csv",
      "start_timestamp": "1685084467",
      "status": "finished",
      "target_table": "demodb.test.targettable",
      "username": "ibmlhadmin",
      "validate_csv_header": false
    }
  ]
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}

          Create an ingestion job

          Create an ingestion job

          POST /ingestion_jobs

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.ingestion_jobs.create

          Request

          Custom Headers

          • AuthInstanceId
            Required*
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Request Body

          Required*
          IngestionJobPrototype

          Request body for submitting ingestion job

          • curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json" -d '{ "target_table": "demodb.test.targettable", "engine_id": "spark123", "execute_config": { "driver_memory": "2G", "driver_cores": 1, "executor_memory": "2G", "executor_cores": 1, "num_executors": 1 }, "source_data_files": "s3://demobucket/data/yellow_tripdata_2022-01.parquet", "source_file_type": "csv", "create_if_not_exist": false, "schema": "[{"type":"int","field_id":0,"header_name":"c1"},{"type":"string","field_id":1,"header_name":"c2"}]", "csv_property": { "field_delimiter": ",", "line_delimiter": "
", "escape_character": " ", "header": true, "encoding": "utf-8" }, "job_id": "ingestion-1699459946935", "username": "user1", }' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/ingestion_jobs"

          Response

          Response Body

          IngestionJob

          Ingestion job

          Status Code

          • 202

            Accepted

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal Server Error

          Example responses
          • {
  "create_if_not_exist": false,
  "csv_property": {
    "encoding": "utf-8",
    "escape_character": "|",
    "field_delimiter": ",",
    "header": true,
    "line_delimiter": "\n"
  },
  "details": "Path does not exist 'demobucket/data/yellow_tripdata_2022-01.parquet'. Detail: [errno 2] No such file or directory",
  "end_timestamp": "1685088775",
  "engine_id": "spark123",
  "engine_name": "sparkdemo",
  "execute_config": {
    "driver_cores": 1,
    "driver_memory": "2G",
    "executor_cores": 1,
    "executor_memory": "2G",
    "num_executors": 1
  },
  "instance_id": "1684432229673971",
  "job_id": "ingestion-1699459946935",
  "partition_by": "col1, col2",
  "schema": "[{\"type\":\"int\",\"field_id\":0,\"header_name\":\"c1\"},{\"type\":\"string\",\"field_id\":1,\"header_name\":\"c2\"}]",
  "source_data_files": "s3://demobucket/data/yellow_tripdata_2022-01.parquet",
  "source_file_type": "csv",
  "start_timestamp": "1685084455",
  "status": "running",
  "target_table": "demodb.test.targettable",
  "username": "ibmlhadmin",
  "valdiate_csv_header": false
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}

          Create an ingestion job for user local files

          Create an ingestion job for user local files

          POST /ingestion_jobs_local_files

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.ingestion_jobs.create

          Request

          Custom Headers

          • AuthInstanceId
            Required*
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Form Parameters

          • source_data_file
            Required*
            binary

            The user local file submitted for ingestion

            Possible values: 0 ≤ length ≤ 524288000

          • target_table
            Required*
            string

            Target table name in format catalog.schema.table

            Possible values: 5 ≤ length ≤ 256, Value must match regular expression ^.*$

          • job_id
            Required*
            string

            Job ID of the job

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-_\\.]+$

          • username
            Required*
            string

            User submitting ingestion job

            Possible values: 1 ≤ length ≤ 256, Value must match regular expression ^.*$

          • source_file_type
            string

            File format of source file

            Allowable values: [csv,parquet,json]

            Possible values: 0 ≤ length ≤ 7, Value must match regular expression ^[a-zA-Z]+

          • csv_property
            string

            Ingestion CSV properties (base64 encoding of a stringifed json).

            Possible values: 0 ≤ length ≤ 256, Value must match regular expression ^.*$

          • create_if_not_exist
            boolean

            Create new target table (if true); Insert into pre-existing target table (if false)

            Default: false

          • validate_csv_header
            boolean

            Validate CSV header if the target table exist.

            Default: false

          • execute_config
            string

            Ingestion engine configuration (base64 encoding of a stringifed json).

            Possible values: 0 ≤ length ≤ 256, Value must match regular expression ^.*$

          • engine_id
            string

            ID of the spark engine to be used for ingestion.

            Possible values: 0 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+$

          • curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: multipart/form-data" -F target_table=demodb.test.table -F create_if_not_exist=true -F job_id=ingestion-1699459946935 -F username=user1 -F source_data_file=@data.csv -F source_file_type=csv -F csv_property=eyJmaWVsZF9kZWxpbWl0ZXIiOiIsIiwibGluZV9kZWxpbWl0ZXIiOiJcbiIsImVzY2FwZV9jaGFyYWN0ZXIiOiJcXCIsImhlYWRlciI6dHJ1ZSwiZW5jb2RpbmciOiJVVEYtOCJ9 "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/ingestion_jobs_local_files"

          Response

          Response Body

          IngestionJob

          Ingestion job

          Status Code

          • 202

            Accepted

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal Server Error

          Example responses
          • {
  "create_if_not_exist": false,
  "csv_property": {
    "encoding": "utf-8",
    "escape_character": "|",
    "field_delimiter": ",",
    "header": true,
    "line_delimiter": "\n"
  },
  "details": "Path does not exist 'demobucket/data/yellow_tripdata_2022-01.parquet'. Detail: [errno 2] No such file or directory",
  "end_timestamp": "1685088775",
  "engine_id": "spark123",
  "engine_name": "sparkdemo",
  "execute_config": {
    "driver_cores": 1,
    "driver_memory": "2G",
    "executor_cores": 1,
    "executor_memory": "2G",
    "num_executors": 1
  },
  "instance_id": "1684432229673971",
  "job_id": "ingestion-1699459946935",
  "partition_by": "col1, col2",
  "schema": "[{\"type\":\"int\",\"field_id\":0,\"header_name\":\"c1\"},{\"type\":\"string\",\"field_id\":1,\"header_name\":\"c2\"}]",
  "source_data_files": "s3://demobucket/data/yellow_tripdata_2022-01.parquet",
  "source_file_type": "csv",
  "start_timestamp": "1685084455",
  "status": "running",
  "target_table": "demodb.test.targettable",
  "username": "ibmlhadmin",
  "valdiate_csv_header": false
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}

          Get ingestion job

          Get a submitted ingestion job

          GET /ingestion_jobs/{job_id}

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.ingestion_job.get

          Request

          Custom Headers

          • AuthInstanceId
            Required*
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Path Parameters

          • job_id
            Required*
            string

            ingestion job id

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_\.]+$

          • curl -X GET -H "accept: application/json" -H "AuthInstanceId: {instance_id}" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/ingestion_jobs/{job_id}"

          Response

          Response Body

          IngestionJob

          Ingestion job

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal Server Error

          Example responses
          • {
  "create_if_not_exist": false,
  "csv_property": {
    "encoding": "utf-8",
    "escape_character": "|",
    "field_delimiter": ",",
    "header": true,
    "line_delimiter": "\n"
  },
  "details": "Path does not exist 'demobucket/data/yellow_tripdata_2022-01.parquet'. Detail: [errno 2] No such file or directory",
  "end_timestamp": "1685088775",
  "engine_id": "spark123",
  "engine_name": "sparkdemo",
  "execute_config": {
    "driver_cores": 1,
    "driver_memory": "2G",
    "executor_cores": 1,
    "executor_memory": "2G",
    "num_executors": 1
  },
  "instance_id": "1684432229673971",
  "job_id": "ingestion-1699459946935",
  "partition_by": "col1, col2",
  "schema": "[{\"type\":\"int\",\"field_id\":0,\"header_name\":\"c1\"},{\"type\":\"string\",\"field_id\":1,\"header_name\":\"c2\"}]",
  "source_data_files": "s3://demobucket/data/yellow_tripdata_2022-01.parquet",
  "source_file_type": "csv",
  "start_timestamp": "1685084455",
  "status": "running",
  "target_table": "demodb.test.targettable",
  "username": "ibmlhadmin",
  "valdiate_csv_header": false
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}

          Delete an ingestion job

          Delete an ingestion job

          DELETE /ingestion_jobs/{job_id}

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.ingestion_jobs.delete

          Request

          Custom Headers

          • AuthInstanceId
            Required*
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Path Parameters

          • job_id
            Required*
            string

            ingestion job id

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_\.]+$

          • curl -X DELETE -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/ingestion_jobs"

          Response

          Status Code

          • 204

            No Content

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal Server Error

          Example responses
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}

          Generate a preview of source file(s)

          Generate a preview of source file(s)

          POST /preview_ingestion_file

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.preview_ingestion_file.create

          Request

          Custom Headers

          • AuthInstanceId
            Required*
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Request Body

          Required*
          PreviewIngestionFilePrototype

          Request body for previewing ingestion source file(s)

          • curl -X POST -H "accept: application/json" -H "AuthInstanceId: {instance_id}" -H "Content-Type: application/json" -d '{ "source_data_files": "s3://demobucket/data/yellow_tripdata_2022-01.parquet", "source_file_type": "csv", "csv_property": { "field_delimiter": ",", "line_delimiter": "
", "escape_character": " ", "header": true, "encoding": "utf-8" }, }' "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/preview_ingestion_file"

          Response

          Response Body

          PreviewIngestionFile

          Schema of the data in the source file

          Status Code

          • 201

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal Server Error

          Example responses
          • {
  "column_names": [
    "col1",
    "col2",
    "col3"
  ],
  "column_types": [
    "int",
    "string",
    "float"
  ],
  "file_name": "s3://demobucket/data/yellow_tripdata_2022-01.parquet",
  "rows": {
    "row_eight": [
      "8",
      "Jane Doe",
      "63.00"
    ],
    "row_five": [
      "5",
      "Jane Doe",
      "57.00"
    ],
    "row_four": [
      "4",
      "Jane Doe",
      "55.00"
    ],
    "row_nine": [
      "9",
      "Jane Doe",
      "65.00"
    ],
    "row_one": [
      "1",
      "Jane Doe",
      "49.00"
    ],
    "row_seven": [
      "7",
      "Jane Doe",
      "61.00"
    ],
    "row_six": [
      "6",
      "Jane Doe",
      "59.00"
    ],
    "row_ten": [
      "10",
      "Jane Doe",
      "67.00"
    ],
    "row_three": [
      "3",
      "Jane Doe",
      "53.00"
    ],
    "row_two": [
      "2",
      "Jane Doe",
      "51.00"
    ]
  }
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}
          • {
  "errors": [
    {
      "code": "bad_request",
      "message": "string",
      "more_info": "string"
    }
  ],
  "exception": "string",
  "message": "string",
  "message_code": "string",
  "more_info": "string",
  "status_code": 0,
  "trace": "cd05b1fc-bc10-4ba6-9a9f-b73c503dbd77"
}

          Register delta or hudi table

          Register delta or hudi table

          POST /catalogs/{catalog_id}/schemas/{schema_id}/register

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.table.register

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Path Parameters

          • catalog_id
            Required*
            string

            catalog id

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+$

          • schema_id
            Required*
            string

            URL encoded schema name

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+

          Request Body

          Required*
          RegisterTableBody

          Request body

          Examples: 
          {
  "metadata_location": "s3a://bucketname/path/to/table/metadata_location/_delta_log",
  "table_name": "mytable"
}
          {
  "metadata_location": "s3a://bucketname/path/to/table/metadata_location/.hoodie",
  "table_name": "mytable"
}
          • curl -X 'POST' 'https://{region}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/register' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}' -H 'Content-Type: application/merge-patch+json' -d '{ 
  "table_name": "table1", 
  "metadata_location": "s3a://bucketname/path/to/table/metadata_location/_delta_log" 
}'

          Response

          Response Body

          RegisterTableCreatedBody

          success response

          Status Code

          • 201

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "response": {
    "message": "Register table success",
    "message_code": "success"
  }
}

          Load table metadata

          Load table metadata

          GET /catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/metadata

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.table.load

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Path Parameters

          • catalog_id
            Required*
            string

            catalog id

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-_]+$

          • schema_id
            Required*
            string

            URL encoded schema name

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+

          • table_id
            Required*
            string

            URL encoded table name

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-%]+

          • curl -X 'GET' 'https://{region}.cp.fyre.ibm.com/lakehouse/api/v2/catalogs/{catalog_id}/schemas/{schema_id}/tables/{table_id}/metadata' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

          Response

          Response Body

          LoadTableResponse

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          No Sample Response

          This method does not specify any sample responses.

          Get all columns

          Get all columns

          GET /columns

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.columns.list

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Query Parameters

          • table_name
            string

            Table name

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+$

          • catalog_name
            string

            Catalog name

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+$

          • schema_name
            string

            Schema name

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+$

          • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/columns?catalog_name={catalog_name}&schema_name={schema_name}&table_name={table_name}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

          Response

          Response Body

          ColumnsResponse

          Column details

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "columns": [
    {
      "bucket": "iceberg-bucket",
      "catalog": "iceberg_data",
      "columns": [
        {
          "column": "club_name",
          "type": "string"
        },
        {
          "column": "location",
          "type": "string"
        },
        {
          "column": "sl_no",
          "type": "int"
        },
        {
          "column": "stadium",
          "type": "string"
        }
      ],
      "owner": "ibmlhadmin",
      "schema": "s3",
      "table": "clubs"
    }
  ],
  "message": "1 entries found",
  "message_code": "success"
}

          Get all schemas for a given catalog

          List Schemas for a catalog with the given catalog_name

          GET /schemas

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.schemas.list

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Query Parameters

          • catalog_name
            string

            Catalog name

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+$

          • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/schemas?catalog_name={catalog_name}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

          Response

          Response Body

          SchemaResponseCollection

          Schema response for list all schemas

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "message": "1 entries found",
  "message_code": "success",
  "schemas": [
    {
      "bucket": "iceberg-bucket",
      "catalog": "iceberg_data",
      "owner": "ibmlhadmin",
      "schema_name": "s3"
    }
  ]
}

          Get schema details

          Get schema details

          GET /schemas/{schema_name}

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.schemadetails.list

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Path Parameters

          • schema_name
            Required*
            string

            Schema name

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+$

          Query Parameters

          • catalog_name
            string

            Catalog name

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+$

          • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/schemas/{schema_name}?catalog_name={catalog_name}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

          Response

          Response Body

          SchemaResponse

          Summary of the schema response

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "message": "1 entry found",
  "message_code": "success",
  "schema": {
    "bucket": "iceberg-bucket",
    "catalog": "iceberg_data",
    "owner": "ibmlhadmin",
    "schema_name": "s3"
  }
}

          Get all tables

          Get all tables

          GET /tables

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.tables.list

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Query Parameters

          • catalog_name
            string

            Catalog name

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+$

          • schema_name
            string

            Schema name

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+$

          • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/tables?catalog_name={catalog_name}&schema_name={schema_name}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

          Response

          Response Body

          TableResponseCollection

          Table response collection

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "message": "1 entries found",
  "message_code": "success",
  "tables": [
    {
      "bucket": "iceberg-bucket",
      "catalog": "iceberg_data",
      "owner": "ibmlhadmin",
      "schema": "s3",
      "table": "clubs"
    }
  ]
}

          Get table details

          Get table details

          GET /tables/{table_name}

          Authorization

          To call this method, you must be assigned one or more IAM access roles that include the following action. You can check your access by going to Users > User > Access.

          • lakehouse.tabledetails.list

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Path Parameters

          • table_name
            Required*
            string

            Table name

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+$

          Query Parameters

          • catalog_name
            string

            Catalog name

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+$

          • schema_name
            string

            Schema name

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\\-]+$

          • curl -X 'GET' 'https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/tables/{table_name}?catalog_name={catalog_name}&schema_name={schema_name}' -H 'accept: application/json' -H 'AuthInstanceId: {instance_id}'

          Response

          Response Body

          TableResponse

          Table response

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "message": "1 entries found",
  "message_code": "success",
  "tables": [
    {
      "bucket": "iceberg-bucket",
      "catalog": "iceberg_data",
      "columns": [
        {
          "column": "club_name",
          "type": "string"
        },
        {
          "column": "sl_no",
          "type": "int"
        }
      ],
      "owner": "ibmlhadmin",
      "schema": "s3",
      "table": "clubs"
    }
  ]
}

          Generate heat or thread dump specific to presto worker or coordinator

          Generate heat or thread dump specific to presto worker or coordinator

          POST /generate_engine_dump

          Request

          Custom Headers

          • AuthInstanceId
            string

            watsonx.data instance ID

            Possible values: 1 ≤ length ≤ 128, Value must match regular expression ^[a-zA-Z0-9\-\:/]+$

          Request Body

          Required*
          EngineDumpProperties

          Request body

          • curl -X POST \-H "accept: application/json" \ 
-H "AuthInstanceId: {token}" \ 
-H "Content-Type: application/json" "https://{cpd-host}.cp.fyre.ibm.com/lakehouse/api/v2/generate_engine_dump" \ 
-d '{  "pod_name": "coordinator-blue-0",  "engine_id": "presto-01",  "dump_type": "heap",  "dump_file_name": "hpi_dump_file"}'

          Response

          Response Body

          GenerateEngineDumpOKBody

          Generate dump creation

          Status Code

          • 200

            Success

          • 400

            Error

          • 401

            Unauthorized

          • 404

            Not found

          • 500

            Internal server error

          Example responses
          • {
  "response": {
    "message": "Dump creation trigerred for the pod <pod name>",
    "status": "success"
  }
}
          id=curlclassName=tab-item-selected
          Privacy statement