Delete assets and their attachments

DELETE /v3/assets

Request

Path Parameters

Query Parameters

project_id

string

The ID of the project to use. space_id, or project_id is required.

space_id

string

The ID of the project to use. space_id, or project_id is required.

asset_ids

Required *

array

A list of asset IDs

asset_type

string

The type of asset

purge_test_data

boolean

Only take effect when delete data_intg_test_case. by default, it is false which means keep data files

Response

200 - The requested operation completed successfully.

202 - The requested operation is in progress.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

Field Match type Example asset.name Starts with ?asset.name=starts:assetName asset.description Equals ?asset.description=starts:assetDesc

To sort the returned results, use one or more of the parameters described in the following section. If no sort key is specified, the results are sorted in descending order on metadata.create_time, returning the most recently created data flows first.

Field	Example
asset.name	?sort=+asset.name
metadata.create_time	?sort=-metadata.create_time

Multiple sort keys can be specified by delimiting them with a comma. For example, to sort in descending order on create_time and then in ascending order on name use: ?sort=-metadata.create_time,+asset.name ">

List assets

GET /v3/assets

Lists the assets that are contained in the specified project.

Use the following parameters to filter the results:

Field	Match type	Example
asset.name	Starts with	?asset.name=starts:assetName
asset.description	Equals	?asset.description=starts:assetDesc

Field	Example
asset.name	?sort=+asset.name
metadata.create_time	?sort=-metadata.create_time

Request

Path Parameters

Query Parameters

project_id

string

The ID of the project to use. space_id, or project_id is required.

space_id

string

The ID of the project to use. space_id, or project_id is required.

asset_type

string

The type of asset

sort

string

The field to sort the results on, including whether to sort ascending (+) or descending (-), for example, sort=-metadata.create_time

start

string

The page token indicating where to start paging from.

limit

integer

The limit of the number of items to return, for example limit=50. If not specified a default of 100 will be used.

asset.name

string

Filter results based on the specified name.

asset.description

string

Filter results based on the specified description.

asset.resource_key

string

Filter results based on the specified resource_key.

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

Create asset

POST /v3/assets

Create asset

Request

Request Body

Asset definition

Path Parameters

Query Parameters

project_id

string

The ID of the project to use. space_id, or project_id is required.

space_id

string

The ID of the project to use. space_id, or project_id is required.

directory_asset_id

string

The directory asset id to create the asset in or move to

Response

201 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - The data source type details cannot be found.

default - An unexpected error occurred. See response for more information.

Create/update asset based on the given zip file.

PUT /v3/assets

Create/update asset based on the given zip file. Unzip the body and use the fixed metadata file(${asset_type}_metadata) to create/update asset, then upload remain files as attachments one by one

Request

Request Body

The zip file to import.

Path Parameters

Query Parameters

project_id

string

The ID of the project to use. space_id, or project_id is required.

space_id

string

The ID of the project to use. space_id, or project_id is required.

asset_type

string

The type of asset

directory_asset_id

string

The directory asset id to create the asset in or move to

override

boolean

whether override if the header file is common path and already exists in the cluster, by default it is false. data_set and file_set only.

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - The data source type details cannot be found.

default - An unexpected error occurred. See response for more information.

Analyze a Flow for Unit Testing

GET /v3/assets/test_cases/{flow_id}

Analyze a DataStage Flow to determine stages and link that need to be stubbed during unit testing.

Request

Path Parameters

flow_id

Required *

string

The id of the flow to analyze

Query Parameters

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

Create Test Case specification from Flow Analysis

POST /v3/assets/test_cases/{flow_id}

Create Test Case specification from Flow Analysis

Request

Request Body

TestCaseAnalysisBody

Flow Analysis

analysis

Required *

TestCaseAnalysis

destinations

array

parameters

object

sources

array

description

string

asset description

name

Required *

string

asset name

Path Parameters

flow_id

Required *

string

The id of the flow to analyze

Query Parameters

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

directory_asset_id

string

The directory asset id to create the asset in or move to

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

Delete asset and their attachments

DELETE /v3/assets/{asset_id}

Delete asset and their attachments

Request

Path Parameters

asset_id

Required *

string

The ID of the asset

Query Parameters

project_id

string

The ID of the project to use. space_id, or project_id is required.

space_id

string

The ID of the project to use. space_id, or project_id is required.

purge_test_data

boolean

Only take effect when delete data_intg_test_case. by default, it is false which means keep data files

Response

200 - The requested operation completed successfully.

202 - The requested operation is in progress.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

Get asset

GET /v3/assets/{asset_id}

Get asset

Request

Path Parameters

asset_id

Required *

string

The ID of the asset

Query Parameters

project_id

string

The ID of the project to use. space_id, or project_id is required.

space_id

string

The ID of the project to use. space_id, or project_id is required.

Response

200 - Asset definition

400 - Bad request. See response for more information.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - The data source type details cannot be found.

500 - An error occurred.

default - An unexpected error occurred. See response for more information.

Update asset with a replacement

PUT /v3/assets/{asset_id}

Update asset with a replacement. If a property is not provided in request body, its corresponding asset field will not be updated.

Request

Request Body

New asset definition

Path Parameters

asset_id

Required *

string

The ID of the asset

Query Parameters

project_id

string

The ID of the project to use. space_id, or project_id is required.

space_id

string

The ID of the project to use. space_id, or project_id is required.

directory_asset_id

string

The directory asset id to create the asset in or move to

override

boolean

whether override if the header file is common path and already exists in the cluster, by default it is false. data_set and file_set only.

Response

200 - The requested operation completed successfully

400 - Bad request. See response for more information.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - The data source type details cannot be found.

default - An unexpected error occurred. See response for more information.

Delete attachments

DELETE /v3/assets/{asset_id}/attachments

Delete attachments

Request

Path Parameters

asset_id

Required *

string

The ID of the asset

Query Parameters

project_id

string

The ID of the project to use. space_id, or project_id is required.

space_id

string

The ID of the project to use. space_id, or project_id is required.

attachment_ids

Required *

array

A list of attachment GUIDs or names

Response

200 - The requested operation completed successfully.

202 - The requested operation is in progress.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

Create attachment

POST /v3/assets/{asset_id}/attachments

Create attachment

Request

Request Body

Attachment content

Path Parameters

asset_id

Required *

string

The ID of the asset

Query Parameters

project_id

string

The ID of the project to use. space_id, or project_id is required.

space_id

string

The ID of the project to use. space_id, or project_id is required.

attachment_name

Required *

string

The name of the new attachment

attachment_type

Required *

string

The mime type of the new attachment

Response

201 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - The data source type details cannot be found.

default - An unexpected error occurred. See response for more information.

Get attachment

GET /v3/assets/{asset_id}/attachments/{attachment_id}

Get attachment

Request

Path Parameters

asset_id

Required *

string

The ID of the asset

attachment_id

Required *

string

The GUID or name of the attachment

Query Parameters

project_id

string

The ID of the project to use. space_id, or project_id is required.

space_id

string

The ID of the project to use. space_id, or project_id is required.

Response

200 - Attachment content

400 - Bad request. See response for more information.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - The data source type details cannot be found.

500 - An error occurred.

default - An unexpected error occurred. See response for more information.

Patch attachment

PATCH /v3/assets/{asset_id}/attachments/{attachment_id}

Patch attachment

Request

Request Body

the patch json

Path Parameters

asset_id

Required *

string

The ID of the asset

attachment_id

Required *

string

The GUID or name of the attachment

Query Parameters

project_id

string

The ID of the project to use. space_id, or project_id is required.

space_id

string

The ID of the project to use. space_id, or project_id is required.

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - The data source type details cannot be found.

default - An unexpected error occurred. See response for more information.

Replace attachment

PUT /v3/assets/{asset_id}/attachments/{attachment_id}

Replace attachment

Request

Request Body

Attachment content

Path Parameters

asset_id

Required *

string

The ID of the asset

attachment_id

Required *

string

The GUID or name of the attachment

Query Parameters

project_id

string

The ID of the project to use. space_id, or project_id is required.

space_id

string

The ID of the project to use. space_id, or project_id is required.

attachment_name

Required *

string

The name of the new attachment

attachment_type

Required *

string

The mime type of the new attachment

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - The data source type details cannot be found.

default - An unexpected error occurred. See response for more information.

Clone asset and its attachments

POST /v3/assets/{asset_id}/clone

Clone asset and its attachments

Request

Path Parameters

asset_id

Required *

string

The ID of the asset

Query Parameters

project_id

string

The ID of the project to use. space_id, or project_id is required.

space_id

string

The ID of the project to use. space_id, or project_id is required.

directory_asset_id

string

The directory asset id to create the asset in or move to

name

string

The new name

override

boolean

whether override if the header file is common path and already exists in the cluster, by default it is false. data_set and file_set only.

Response

201 - The requested operation completed successfully.

400 - Bad request. See response for more information.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - The data source type details cannot be found.

default - An unexpected error occurred. See response for more information.

Export the asset with its all attachments as a zip

GET /v3/assets/{asset_id}/export

Export the asset with its all attachments as a zip which include the fixed metadata file(${asset_type}_metadata) and all attachments(without entity-attachment)

Request

Path Parameters

asset_id

Required *

string

The ID of the asset

Query Parameters

project_id

string

The ID of the project to use. space_id, or project_id is required.

space_id

string

The ID of the project to use. space_id, or project_id is required.

max_allowed_data_size

integer

(Optional) Only the data size smaller than this are exported. In mega byte, only take effect on data_intg_data_set and data_intg_file_set and data_intg_test_case. By default or set to negative, means 500. If set 0, skip data. if greater than 1000, use 1000.

Response

200 - The requested operation completed successfully.

400 - Bad request. See response for more information.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - The data source type details cannot be found.

500 - An error occurred.

default - An unexpected error occurred. See response for more information.

Delete DataStage flows

DELETE /v3/data_intg_flows

Deletes the specified data flows in a project or catalog (either `project_id` or `catalog_id` must be set).

If the deletion of the data flows and their runs will take some time to finish, then a 202 response will be returned and the deletion will continue asynchronously.

All the data flow runs associated with the data flows will also be deleted. If a data flow is still running, it will not be deleted unless the force parameter is set to true. If a data flow is still running and the force parameter is set to true, the call returns immediately with a 202 response. The related data flows are deleted after the data flow runs are stopped.

Request

Path Parameters

Query Parameters

id

Required *

array

The list of DataStage flow IDs to delete.

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

force

boolean

Whether to stop all running data flows. Running DataStage flows must be stopped before the DataStage flows can be deleted.

Response

202 - The requested operation is in progress.

204 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Field Match type Example entity.name Equals entity.name=MyDataStageFlow entity.name Starts with entity.name=starts:MyData entity.description Equals entity.description=movement entity.description Starts with entity.description=starts:data

To sort the results, use one or more of the parameters described in the following section. If no sort key is specified, the results are sorted in descending order on `metadata.create_time` (i.e. returning the most recently created data flows first).

Field	Example
sort	`sort=+entity.name` (sort by ascending name)
sort	`sort=-metadata.create_time` (sort by descending creation time)

Multiple sort keys can be specified by delimiting them with a comma. For example, to sort in descending order on `create_time` and then in ascending order on name use: `sort=-metadata.create_time`,`+entity.name` ">

Get metadata for DataStage flows

GET /v3/data_intg_flows

Lists the metadata and entity for DataStage flows that are contained in the specified project.

Use the following parameters to filter the results:

Field	Match type	Example
`entity.name`	Equals	`entity.name=MyDataStageFlow`
`entity.name`	Starts with	`entity.name=starts:MyData`
`entity.description`	Equals	`entity.description=movement`
`entity.description`	Starts with	`entity.description=starts:data`

Field	Example
sort	`sort=+entity.name` (sort by ascending name)
sort	`sort=-metadata.create_time` (sort by descending creation time)

Request

Path Parameters

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

sort

string

The field to sort the results on, including whether to sort ascending (+) or descending (-), for example, sort=-metadata.create_time.

start

string

The page token indicating where to start paging from.

limit

integer

The limit of the number of items to return for each page, for example limit=50. If not specified a default of 100 will be used. The maximum value of limit is 200.

entity.name

string

Filter results based on the specified name.

entity.description

string

Filter results based on the specified description.

Response

200 - The requested operation completed successfully.

data_flows

array

A page from a collection of DataStage flows.

first

HrefModel

URI of a resource.

last

HrefModel

URI of a resource.

limit

integer

The number of data flows requested to be returned.

HrefModel

URI of a resource.

HrefModel

URI of a resource.

total_count

integer

The total number of DataStage flows available.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Create DataStage flow

POST /v3/data_intg_flows

Creates a DataStage flow in the specified project or catalog (either `project_id` or `catalog_id` must be set). All subsequent calls to use the data flow must specify the project or catalog ID the data flow was created in.

Request

Request Body

PipelineFlow

Pipeline json to be attached.

pipeline_flows

PipelineJson

Pipeline flow to be stored.

app_data

object

Object containing app-specific data.

doc_type

string

The document type.

external_paramsets

array

Array of parameter set references.

id

string

Document identifier, GUID recommended.

json_schema

string

Refers to the JSON schema used to validate documents of this type.

parameters

object

Parameters for the flow document.

pipelines

array

Array of pipeline.

primary_pipeline

string

Reference to the primary (main) pipeline flow within the document.

runtimes

array

Runtime information for pipeline flow.

schemas

array

Array of data record schemas used in the pipeline.

version

string

Pipeline flow version.

Path Parameters

Query Parameters

data_intg_flow_name

Required *

string

The data flow name.

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

directory_asset_id

string

The directory asset ID.

Response

201 - The requested operation completed successfully.

attachments

array

Metadata information for datastage flow.

entity

DataIntgFlowEntity

The underlying DataStage flow definition.

metadata

AssetSystemMetadata

System metadata about an asset.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Delete DataStage subflows

DELETE /v3/data_intg_flows/subflows

Deletes the specified data subflows in a project or catalog (either `project_id` or `catalog_id` must be set).

If the deletion of the data subflows will take some time to finish, then a 202 response will be returned and the deletion will continue asynchronously.

Request

Path Parameters

Query Parameters

id

Required *

array

The list of DataStage subflow IDs to delete.

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

Response

202 - The requested operation is in progress.

204 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Field Match type Example entity.name Equals entity.name=MyDataStageSubFlow entity.name Starts with entity.name=starts:MyData entity.description Equals entity.description=movement entity.description Starts with entity.description=starts:data

Field	Example
sort	`sort=+entity.name` (sort by ascending name)
sort	`sort=-metadata.create_time` (sort by descending creation time)

Get metadata for DataStage subflows

GET /v3/data_intg_flows/subflows

Lists the metadata and entity for DataStage subflows that are contained in the specified project.

Use the following parameters to filter the results:

Field	Match type	Example
`entity.name`	Equals	`entity.name=MyDataStageSubFlow`
`entity.name`	Starts with	`entity.name=starts:MyData`
`entity.description`	Equals	`entity.description=movement`
`entity.description`	Starts with	`entity.description=starts:data`

Field	Example
sort	`sort=+entity.name` (sort by ascending name)
sort	`sort=-metadata.create_time` (sort by descending creation time)

Request

Path Parameters

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

sort

string

The field to sort the results on, including whether to sort ascending (+) or descending (-), for example, sort=-metadata.create_time.

start

string

The page token indicating where to start paging from.

limit

integer

The limit of the number of items to return for each page, for example limit=50. If not specified a default of 100 will be used. The maximum value of limit is 200.

entity.name

string

Filter results based on the specified name.

entity.description

string

Filter results based on the specified description.

Response

200 - The requested operation completed successfully.

data_flows

array

A page from a collection of DataStage flows.

first

HrefModel

URI of a resource.

last

HrefModel

URI of a resource.

limit

integer

The number of data flows requested to be returned.

HrefModel

URI of a resource.

HrefModel

URI of a resource.

total_count

integer

The total number of DataStage flows available.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Create DataStage subflow

POST /v3/data_intg_flows/subflows

Creates a DataStage subflow in the specified project or catalog (either `project_id` or `catalog_id` must be set). All subsequent calls to use the data flow must specify the project or catalog ID the data flow was created in.

Request

Request Body

SubFlowUpdate

Pipeline json to be attached.

entity

SubFlowEntityJson

data_rule

object

The JSON object details will be saved as entity of the subflow.

subtype

string

The sub type of the subflow.

pipeline_flows

PipelineJson

Pipeline flow to be stored.

app_data

object

Object containing app-specific data.

doc_type

string

The document type.

external_paramsets

array

Array of parameter set references.

id

string

Document identifier, GUID recommended.

json_schema

string

Refers to the JSON schema used to validate documents of this type.

parameters

object

Parameters for the flow document.

pipelines

array

Array of pipeline.

primary_pipeline

string

Reference to the primary (main) pipeline flow within the document.

runtimes

array

Runtime information for pipeline flow.

schemas

array

Array of data record schemas used in the pipeline.

version

string

Pipeline flow version.

Path Parameters

Query Parameters

data_intg_subflow_name

Required *

string

The DataStage subflow name.

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

directory_asset_id

string

The directory asset ID.

Response

201 - The requested operation completed successfully.

attachments

array

Metadata information for datastage flow.

entity

DataIntgFlowEntity

The underlying DataStage flow definition.

metadata

AssetSystemMetadata

System metadata about an asset.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Get DataStage subflow

GET /v3/data_intg_flows/subflows/{data_intg_subflow_id}

Lists the DataStage subflow that is contained in the specified project. Attachments, metadata and a limited number of attributes from the entity of each DataStage flow is returned.

Request

Path Parameters

data_intg_subflow_id

Required *

string

The DataStage subflow ID to use.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

Response

200 - The requested operation completed successfully.

attachments

PipelineJson

Pipeline flow to be stored.

entity

DataIntgFlowEntity

The underlying DataStage flow definition.

metadata

AssetSystemMetadata

System metadata about an asset.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

500 - Unexpected error.

Update DataStage subflow

PUT /v3/data_intg_flows/subflows/{data_intg_subflow_id}

Modifies a data subflow in the specified project or catalog (either `project_id` or `catalog_id` must be set). All subsequent calls to use the data flow must specify the project or catalog ID the data flow was created in.

Request

Request Body

SubFlowUpdate

Pipeline json to be attached.

entity

SubFlowEntityJson

data_rule

object

The JSON object details will be saved as entity of the subflow.

subtype

string

The sub type of the subflow.

pipeline_flows

PipelineJson

Pipeline flow to be stored.

app_data

object

Object containing app-specific data.

doc_type

string

The document type.

external_paramsets

array

Array of parameter set references.

id

string

Document identifier, GUID recommended.

json_schema

string

Refers to the JSON schema used to validate documents of this type.

parameters

object

Parameters for the flow document.

pipelines

array

Array of pipeline.

primary_pipeline

string

Reference to the primary (main) pipeline flow within the document.

runtimes

array

Runtime information for pipeline flow.

schemas

array

Array of data record schemas used in the pipeline.

version

string

Pipeline flow version.

Path Parameters

data_intg_subflow_id

Required *

string

The DataStage subflow ID to use.

Query Parameters

data_intg_subflow_name

Required *

string

The DataStage subflow name.

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

directory_asset_id

string

The directory asset ID.

Response

201 - The requested operation completed successfully.

attachments

array

Metadata information for datastage flow.

entity

DataIntgFlowEntity

The underlying DataStage flow definition.

metadata

AssetSystemMetadata

System metadata about an asset.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Modifies attributes of DataStage subflow

PUT /v3/data_intg_flows/subflows/{data_intg_subflow_id}/attributes

Modifies attributes of a data subflow in the specified project or catalog (either `project_id` or `catalog_id` must be set).

Request

Request Body

FlowAttributes

attributes of subflows to modify.

description

string

description of the asset.

directory_asset_id

string

The directory asset ID of the asset.

name

string

name of the asset.

Path Parameters

data_intg_subflow_id

Required *

string

The DataStage subflow ID to use.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

Response

201 - The requested operation completed successfully.

attachments

array

Metadata information for datastage flow.

entity

DataIntgFlowEntity

The underlying DataStage flow definition.

metadata

AssetSystemMetadata

System metadata about an asset.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Clone DataStage subflow

POST /v3/data_intg_flows/subflows/{data_intg_subflow_id}/clone

Create a DataStage subflow in the specified project or catalog based on an existing DataStage subflow in the same project or catalog.

Request

Path Parameters

data_intg_subflow_id

Required *

string

The DataStage subflow ID to use.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

directory_asset_id

string

The directory asset ID.

data_intg_subflow_name

string

The data subflow name.

Response

201 - The requested operation completed successfully.

attachments

array

Metadata information for datastage flow.

entity

DataIntgFlowEntity

The underlying DataStage flow definition.

metadata

AssetSystemMetadata

System metadata about an asset.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Get DataStage flow

GET /v3/data_intg_flows/{data_intg_flow_id}

Lists the DataStage flow that is contained in the specified project. Attachments, metadata and a limited number of attributes from the entity of each DataStage flow is returned.

Request

Path Parameters

data_intg_flow_id

Required *

string

The DataStage flow ID to use.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

Response

200 - The requested operation completed successfully.

attachments

PipelineJson

Pipeline flow to be stored.

entity

DataIntgFlowEntity

The underlying DataStage flow definition.

metadata

AssetSystemMetadata

System metadata about an asset.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

500 - Unexpected error.

Update DataStage flow

PUT /v3/data_intg_flows/{data_intg_flow_id}

Modifies a data flow in the specified project or catalog (either `project_id` or `catalog_id` must be set). All subsequent calls to use the data flow must specify the project or catalog ID the data flow was created in.

Request

Request Body

PipelineFlow

Pipeline json to be attached.

pipeline_flows

PipelineJson

Pipeline flow to be stored.

app_data

object

Object containing app-specific data.

doc_type

string

The document type.

external_paramsets

array

Array of parameter set references.

id

string

Document identifier, GUID recommended.

json_schema

string

Refers to the JSON schema used to validate documents of this type.

parameters

object

Parameters for the flow document.

pipelines

array

Array of pipeline.

primary_pipeline

string

Reference to the primary (main) pipeline flow within the document.

runtimes

array

Runtime information for pipeline flow.

schemas

array

Array of data record schemas used in the pipeline.

version

string

Pipeline flow version.

Path Parameters

data_intg_flow_id

Required *

string

The DataStage flow ID to use.

Query Parameters

data_intg_flow_name

Required *

string

The data flow name.

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

directory_asset_id

string

The directory asset ID.

Response

201 - The requested operation completed successfully.

attachments

array

Metadata information for datastage flow.

entity

DataIntgFlowEntity

The underlying DataStage flow definition.

metadata

AssetSystemMetadata

System metadata about an asset.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Modifies attributes of a DataStage flow

PUT /v3/data_intg_flows/{data_intg_flow_id}/attributes

Modifies attributes of a DataStage flow in the specified project or catalog (either `project_id` or `catalog_id` must be set).

Request

Request Body

FlowAttributes

attributes of flow to modify.

description

string

description of the asset.

directory_asset_id

string

The directory asset ID of the asset.

name

string

name of the asset.

Path Parameters

data_intg_flow_id

Required *

string

The DataStage flow ID to use.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

Response

201 - The requested operation completed successfully.

attachments

array

Metadata information for datastage flow.

entity

DataIntgFlowEntity

The underlying DataStage flow definition.

metadata

AssetSystemMetadata

System metadata about an asset.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Clone DataStage flow

POST /v3/data_intg_flows/{data_intg_flow_id}/clone

Create a DataStage flow in the specified project or catalog or space based on an existing DataStage flow in the same project or catalog or space.

Request

Path Parameters

data_intg_flow_id

Required *

string

The DataStage flow ID to use.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

directory_asset_id

string

The directory asset ID.

data_intg_flow_name

string

The data flow name.

Response

201 - The requested operation completed successfully.

attachments

array

Metadata information for datastage flow.

entity

DataIntgFlowEntity

The underlying DataStage flow definition.

metadata

AssetSystemMetadata

System metadata about an asset.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Compile DataStage flow to generate runtime assets

POST /v3/ds_codegen/compile/{data_intg_flow_id}

Generate the runtime assets for a DataStage flow in the specified project or catalog for a specified runtime type. Either project_id or catalog_id must be specified.

Request

Path Parameters

data_intg_flow_id

Required *

string

The DataStage flow ID to use.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

runtime_type

string

The type of the runtime to use. e.g. dspxosh or Spark etc. If not provided queried from within pipeline flow if available otherwise default of dspxosh is used.

enable_sql_pushdown

boolean

Whether to enable the SQL pushdown code generation or not. When this flag is set to true and enable_pushdown_source is not specified, enable_pushdown_source will be set to true. When this flag is set to true and enable_pushdown_target is not specified, enable_pushdown_target will be set to true.

enable_async_compile

boolean

Whether to compile the flow asynchronously or not. When set to true, the compile request will be queued and then compiled. Response will be returned immediately as "Compiling". For compile status, call get compile status api.

enable_pushdown_source

boolean

Whether to enable the push sql to source connectors. Setting this flag to true will automatically set enable_sql_pushdown to true if the latter is not specified or is explicitly set to false. When this flag is set to true and enable_push_processing_to_source is not specified, enable_push_processing_to_source will be automatically set to true as well. When this flag is set to true and enable_push_join_to_source is not speicified, enable_push_join_to_source will be automatically set to true as well.

enable_push_processing_to_source

boolean

Whether to enable pushing processing stages to source connectors or not. Setting this flag to true will automatically set enable_pushdown_source to true if the latter is not specified or is explicitly set to false.

enable_push_join_to_source

boolean

Whether to enable pushing join/lookup stages to source connectors or not. Setting this flag to true will automatically set enable_pushdown_source to true if the latter is not specified or is explicitly set to false.

enable_pushdown_target

boolean

Whether to enable the push sql to target connectors. Setting this flag to true will automatically set enable_sql_pushdown to true if the latter is not specified or is explicitly set to false.

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

422 - Request object contains invalid information. Server is not able to process the request object.

500 - Unexpected error.

Generate OPD-code for DataStage buildop

POST /v3/ds_codegen/generateBuildOp/{data_intg_bldop_id}

Generate the runtime assets for a DataStage buildop in the specified project or catalog for a specified runtime type. Either project_id or catalog_id must be specified.

Request

Request Body

Buildop

BuildOp json to be attached.

build

object

Build info

interfaces

object

null

logic

object

Operator business logic

creator

object

Creator information.

author

string

Author name

vendor

string

Vendor name

version

string

Version

directory_asset

object

directory information.

general

object

General information

class_name

string

Class name

command

string

Command name

execmode

string

Exec Mode

node_type_name

string

Node type name

operator_name

string

Operator name

wrapped_name

string

Wrapped name

properties

array

List of stage properties

schemas

array

Array of data record schemas used in the buildop.

type

string

The operator type.

ui_data

object

UI data.

wrapped

object

Wrapped info

environment

object

Environment information

interfaces

object

Interfaces

Path Parameters

data_intg_bldop_id

Required *

string

The DataStage BuildOp-Asset-ID to use.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

runtime_type

string

The type of the runtime to use. e.g. dspxosh or Spark etc. If not provided queried from within pipeline flow if available otherwise default of dspxosh is used.

enable_async_compile

boolean

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

422 - Request object contains invalid information. Server is not able to process the request object.

500 - Unexpected error.

Delete pipeline cache

DELETE /v3/ds_codegen/pipeline/cache/{pipeline_id}

Permanently remove all optimized runner cache (if any) for a Watson pipeline in the specified project or catalog for a specified runtime type. Either project_id or catalog_id must be specified.

Request

Path Parameters

pipeline_id

Required *

string

The Watson Pipeline ID to use.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

cascade

boolean

For Compile Watson pipeline API, whether or not to compile the primary(main) pipeline and all its nested pipelines. When the flag is set to false, only the primary pipeline will be compiled. The flag is set to true by default. For Delete pipeline cache API, whether or not to delete pipeline cache for all nested pipelines referenced by the pipeline. When this flag is set to true , the entire cache for the pipeline and all nested pipelines referenced in this pipeline will be permanently removed. The flag is set to false by default.

Response

204 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

500 - Unexpected error.

Compile Watson pipeline to generate runtime code

POST /v3/ds_codegen/pipeline/compile/{pipeline_id}

Generate Runtime code for a Watson pipeline in the specified project or catalog for a specified runtime type. Either project_id or catalog_id must be specified.

Request

Path Parameters

pipeline_id

Required *

string

The Watson Pipeline ID to use.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

enable_inline_pipeline

boolean

Whether to enable inline pipeline execution or not. When this flag is set to true, no individual job runs will be created for nested pipelines. The flag is set to false by default.

runtime_type

string

The type of the runtime to use. e.g. dspxosh or Spark etc. If not provided queried from within pipeline flow if available otherwise default of dspxosh is used.

job_name_suffix

string

The name suffix for the created job, will use the pipeline name suffix configured in datastage project settings.

enable_caching

boolean

Whether to enable pipeline caching or not. Caching is disabled by default.

enable_versioning

boolean

Whether to enable pipeline versioning or not. Versioning is disabled by default.

cascade

boolean

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

422 - Request object contains invalid information. Server is not able to process the request object.

500 - Unexpected error.

Retrieve optimized pipeline info

GET /v3/ds_codegen/pipeline/info/{pipeline_id}

Retrieve pipeline info including generated python code, version, last compile timestamp, and cached results.

Request

Path Parameters

pipeline_id

Required *

string

The Watson Pipeline ID to use.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

Response

200 - The requested operation completed successfully.

cache

OptimizedPipelineInfoCache

Information about the optimized pipeline cache.

compilation

OptimizedPipelineInfoCompilation

Information about the optimized pipeline compilation.

name

string

Pipeline name.

version

string

Build version of the optimized runner compiler

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Get the status of a previous git commit request

GET /v3/migration/git_commit

Gets the status of a git commit request. The status field in the response object indicates if the given commit is completed, in progress, or failed. Detailed status information about the commit progress is also contained in the response object.

Request

Path Parameters

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` is required.

project_id

string

The ID of the project to use. `project_id` or `catalog_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - Status of the git commit request cannot be found. This can be due to the given project is not valid, or the git commit has been completed long ago and its status information is no longer available.

default - An error occurred. See response for more information.

Import Datastage components from git.

POST /v3/migration/git_pull

Creates DataStage components from the git. This is an asynchronous call. The API call returns almost immediately which does not necessarily imply the completion of the import request. It only means that the import request has been accepted. The status field of the import request is included in the import response object. The status "completed" ("in_progress", "failed", resp.) indicates the import request is completed (in progress, and failed, resp.) The job export file for an import request may contain one mor more data flows. Unless the on_failure option is set to "stop", a completed import request may contain not only successfully imported data flows but also data flows that cannot be imported.

Request

Request Body

asset list

Path Parameters

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` is required.

project_id

string

The ID of the project to use. `project_id` or `catalog_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

git_repo

string

The name of the git repo to use.

git_branch

string

The name of the git branch to use.

git_folder

string

The name of the git folder where the project contents are committed/fetched.

git_tag

string

The name of the git tag to use.

on_failure

string

Action when the first import failure occurs. The default action is "continue" which will continue importing the remaining data flows. The "stop" action will stop the import operation upon the first error.

conflict_resolution

string

Resolution when data flow to be imported has a name conflict with an existing data flow in the project or catalog. The default conflict resolution is "skip" will skip the data flow so that it will not be imported. The "rename" resolution will append "_Import_NNNN" suffix to the original name and use the new name for the imported data flow, while the "replace" resolution will first remove the existing data flow with the same name and import the new data flow. For the "rename_replace" option, when the flow name is already used, a new flow name with the suffix "_DATASTAGE_ISX_IMPORT" will be used. If the name is not currently used, the imported flow will be created with this name. In case the new name is already used, the existing flow will be removed first before the imported flow is created. With the rename_replace option, job creation will be determined as follows. If the job name is already used, a new job name with the suffix ".DataStage job" will be used. If the new job name is not currently used, the job will be created with this name. In case the new job name is already used, the job creation will not happen and an error will be raised.

enable_notification

boolean

enable/disable notification. Default value is true.

import_only

boolean

Skip flow compilation.

include_dependencies

boolean

If set to false, skip dependencies, only import flow. If not specified or set to true, import flow and dependencies.

asset_type

string

Asset types separated by commas (','). Only assets of these types will be imported. If not specified, all supported assets will be imported.

skip_dependencies

string

Skip dependencies separated by commas (','), it is only used with replace option. If specified the skiped dependencies, only assets of these types will be skiped if the assets are existed.

replace_mode

string

This parameter takes effect when conflict_resolution is set to replace or skip. soft- merge the parameter set, add new parameter only, keep the old value sets hard- replace all

import_binaries

boolean

Import binaries

project_folder

string

The name of the project folder you would like to pull assets from.

project_folder_recursive

boolean

If true, will pull all subfolders of the project being pulled

Response

202 - The requested import operation has been accepted. However, the import operation may or may not be completed.

The status field in the import response object describes the current status of the import. The response "Location" header provides a convenient url for retrieving the status with a GET request.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Cancel a previous import request

DELETE /v3/migration/git_pull/{import_id}

Cancel a previous import request.

Request

Path Parameters

import_id

Required *

string

Unique ID of the import request.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` is required.

project_id

string

The ID of the project to use. `project_id` or `catalog_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

Response

204 - The import cancellation request was accepted.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - Status of the git pull request cannot be found. This can be due to the given import_id is not valid, or the git pull has been completed long ago and its status information is no longer available.

default - An error occurred. See response for more information.

Get the status of a previous git pull request

GET /v3/migration/git_pull/{import_id}

Gets the status of a git pull request. The status field in the response object indicates if the given import is completed, in progress, or failed. Detailed status information about each imported data flow is also contained in the response object.

Request

Path Parameters

import_id

Required *

string

Unique ID of the pull request.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` is required.

project_id

string

The ID of the project to use. `project_id` or `catalog_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

format

string

format of isx import report

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - Status of the git pull request cannot be found. This can be due to the given import_id is not valid, or the git pull has been completed long ago and its status information is no longer available.

default - An error occurred. See response for more information.

Get the status of a git repo

GET /v3/migration/git_status

Gets the status on objects changed in git repo with respect to a project. Detailed status information about changes committed in a project as well as from git.

Request

Path Parameters

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` is required.

project_id

string

The ID of the project to use. `project_id` or `catalog_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

git_repo

string

The name of the git repo to use.

git_branch

string

The name of the git branch to use.

git_tag

string

The name of the git tag to use.

git_folder

string

The name of the git folder where the project contents are committed/fetched.

format

string

format of isx import report

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - Status of the differences between a project and git repo.

default - An error occurred. See response for more information.

Create V3 data flows from the attached job export file

POST /v3/migration/isx_imports

Creates data flows from the attached job export file. This is an asynchronous call. The API call returns almost immediately which does not necessarily imply the completion of the import request. It only means that the import request has been accepted. The status field of the import request is included in the import response object. The status "completed" ("in_progress", "failed", resp.) indicates the import request is completed (in progress, and failed, resp.) The job export file for an import request may contain one mor more data flows. Unless the on_failure option is set to "stop", a completed import request may contain not only successfully imported data flows but also data flows that cannot be imported.

Request

Request Body

The ISX file to import. The maximum file size is 1GB.

Path Parameters

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` is required.

project_id

string

The ID of the project to use. `project_id` or `catalog_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

on_failure

string

conflict_resolution

string

attachment_type

string

Type of attachment. The default attachment type is "isx".

file_name

string

Name of the input file, if it exists.

enable_notification

boolean

enable/disable notification. Default value is true.

import_only

boolean

Skip flow compilation.

create_missing_parameters

boolean

Create missing parameter sets and job parameters.

enable_rulestage_integration

boolean

enable/disable wkc rule stage migration. Default value is false.

enable_local_connection

boolean

enable local connection migration. Default value is false.

asset_type

string

Asset types separated by commas (','). Only assets of these types will be imported. If not specified, all supported assets will be imported.

create_connection_parametersets

boolean

Create generic parameter sets for default connection values migration. Default value is true.

storage_path

string

Folder path of the storage volume for routine scripts and other data assets.

replace_mode

string

This parameter takes effect when conflict_resolution is set to replace or skip. soft- merge the parameter set, add new parameter only, keep the old value sets hard- replace all

migrate_to_platform_connection

boolean

Will migrate all isx connections to available platform connections instead of their datastage optimized versions.

use_dsn_name

boolean

Enables private cloud migration of ODBC connector 'datasource' to 'dsn_name' instead of generating parameter references. Default is false.

migrate_to_send_email

boolean

Will migrate all notification activitie stages in sequence job to send email task nodes.

enable_folder

boolean

Enable folder support.

migrate_hive_impala

boolean

Will migrate hive isx connections to impala.

from

string

Migrate from which stage.

to

string

Migrate to which stage.

job_name_with_invocation_id

boolean

whether to migrate $JobName to ${jobName}.${DSInvocationJobId} if invocation id present. Default value is false.

annotation_styling

string

format of annotation styling

migrate_to_datastage_division

boolean

migrate the division operator in any expression to datastage division ds.DIV.

run_job_by_name

boolean

run the datastage/pipeline job by job name, default is by id.

enable_optimized_pipeline

boolean

Enable optimized pipeline. Default value is false.

use_jinja_template

boolean

Use jinja template in bash script, all environment vars will be invoked as "" instead of "${env_var}" in bash script. Default value is false.

enable_flow_autosave

boolean

Enable flow autosave or not. Default value is false.

migrate_jdbc_impala

boolean

Will migrate JDBC Impala isx connections to impala.

enable_inline_pipeline

boolean

enable inline mode for the pipeline compilation.

optimized_job_name_suffix

string

Optimized sequence job name suffix, project default setting will take effect if not specified.

migrate_bash_param

boolean

migrate bash script parameter or not for pipeline.

is_define_parameter_toggled

boolean

Prompt runtime parameters settings before running a flow.

migrate_stp_plugin

boolean

Will migrate STPPX Teradata to Teradata Plugin

migrate_userstatus

boolean

Will migrate setuserstatus calls to DSSetUserStatus bash function

skip_connections

boolean

Applies ONLY when --conflict-resolution replace is selected. When this flag is set we will NOT create new connections for the existing flows that are being remigrated but keep the connection links that are already present in the existing flows.

Response

202 - The requested import operation has been accepted. However, the import operation may or may not be completed.

The status field in the import response object describes the current status of the import. The response "Location" header provides a convenient url for retrieving the status with a GET request.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Cancel a previous import request

DELETE /v3/migration/isx_imports/{import_id}

Cancel a previous import request. Use GET /v3/migration/imports/{import_id} to obtain the current status of the import, including whether it has been cancelled.

Request

Path Parameters

import_id

Required *

string

Unique ID of the import request.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` is required.

project_id

string

The ID of the project to use. `project_id` or `catalog_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

Response

204 - The import cancellation request was accepted.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - Status of the import request cannot be found. This can be due to the given import_id is not valid, or the import has been completed long ago and its status information is no longer available.

default - An error occurred. See response for more information.

Get the status of a previous import request

GET /v3/migration/isx_imports/{import_id}

Gets the status of an import request. The status field in the response object indicates if the given import is completed, in progress, or failed. Detailed status information about each imported data flow is also contained in the response object.

Request

Path Parameters

import_id

Required *

string

Unique ID of the import request.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` is required.

project_id

string

The ID of the project to use. `project_id` or `catalog_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

format

string

format of isx import report

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - Status of the import request cannot be found. This can be due to the given import_id is not valid, or the import has been completed long ago and its status information is no longer available.

default - An error occurred. See response for more information.

Get Project and cache to reduce rate limit issue.

GET /v3/migration/project_info

Get Project id from name, should leverage ds-common-services to cache more information.

Request

Path Parameters

Query Parameters

project_name

Required *

string

Name of the project/space.

is_space

boolean

Is Space.

Response

200 - Success.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

export flows with dependencies as a zip file.

POST /v3/migration/zip_exports

export flows with dependencies as a zip file.

Request

Request Body

flows and dependencies metadata

Path Parameters

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` is required.

project_id

string

The ID of the project to use. `project_id` or `catalog_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

remove_secrets

boolean

remove secrets from exported flows, default value is false.

include_dependencies

boolean

include dependencies. If no dependencies are specified in the payload, all dependencies will be included.

id

array

The list of flow IDs to export.

type

string

Type of flow. The default flow type is "data_intg_flow". It is only used with 'id' parameter.

include_data_assets

boolean

include data_assets. If set as true, all referenced data_assets will be included.

exclude_data_files

boolean

skip the actual data files for datastage dataset and fileset when exporting flows as zip.

asset_type_filter

array

Filter assets to be exported by asset type

export_binaries

boolean

Export binaries

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Create data flows from zip file

POST /v3/migration/zip_imports

Request

Request Body

The zip file to import. The maximum file size is 1GB.

Path Parameters

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` is required.

project_id

string

The ID of the project to use. `project_id` or `catalog_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

on_failure

string

conflict_resolution

string

file_name

string

Name of the input file, if it exists.

enable_notification

boolean

enable/disable notification. Default value is true.

import_only

boolean

Skip flow compilation.

include_dependencies

boolean

If set to false, skip dependencies, only import flow. If not specified or set to true, import flow and dependencies.

asset_type

string

Asset types separated by commas (','). Only assets of these types will be imported. If not specified, all supported assets will be imported.

skip_dependencies

string

Skip dependencies separated by commas (','), it is only used with replace option. If specified the skiped dependencies, only assets of these types will be skiped if the assets are existed.

replace_mode

string

This parameter takes effect when conflict_resolution is set to replace or skip. soft- merge the parameter set, add new parameter only, keep the old value sets hard- replace all

import_binaries

boolean

Import binaries

run_job_by_name

boolean

run the datastage/pipeline job by job name, default is by id.

enable_optimized_pipeline

boolean

Enable optimized pipeline. Default value is false.

enable_inline_pipeline

boolean

enable inline mode for the pipeline compilation.

optimized_job_name_suffix

string

Optimized sequence job name suffix, project default setting will take effect if not specified.

is_define_parameter_toggled

boolean

Prompt runtime parameters settings before running a flow.

override_dataset

boolean

This parameter takes effect when conflict_resolution is set to replace for data_set and file_set. by defaut it is false, break importing if the header file is a common path and already exists in the cluster, otherwise force override it

Response

202 - The requested import operation has been accepted. However, the import operation may or may not be completed.

The status field in the import response object describes the current status of the import. The response "Location" header provides a convenient url for retrieving the status with a GET request.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Cancel a previous import request

DELETE /v3/migration/zip_imports/{import_id}

Cancel a previous import request.

Request

Path Parameters

import_id

Required *

string

Unique ID of the import request.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` is required.

project_id

string

The ID of the project to use. `project_id` or `catalog_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

Response

204 - The import cancellation request was accepted.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - Status of the import request cannot be found. This can be due to the given import_id is not valid, or the import has been completed long ago and its status information is no longer available.

default - An error occurred. See response for more information.

Get the status of a previous import request

GET /v3/migration/zip_imports/{import_id}

Request

Path Parameters

import_id

Required *

string

Unique ID of the import request.

Query Parameters

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` is required.

project_id

string

The ID of the project to use. `project_id` or `catalog_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

format

string

format of isx import report

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - Status of the import request cannot be found. This can be due to the given import_id is not valid, or the import has been completed long ago and its status information is no longer available.

default - An error occurred. See response for more information.

List all Standardization Rules in a project.

GET /v3/quality_stage/rules

List all Standardization Rules in a project.

Request

Path Parameters

Query Parameters

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Removes the Standardization Rule from a project.

DELETE /v3/quality_stage/rules/{rule_name}

Removes the Standardization Rule from a project. For the built-in one, it will restore to the defaults and do not actually delete it from the system.

Request

Path Parameters

rule_name

Required *

string

The rule name or the asset id.

Query Parameters

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

location

Required *

string

The location of rule set. Required only when rule_name is not an asset id

Response

204 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Get basic information for the Standardization Rule

GET /v3/quality_stage/rules/{rule_name}

Get basic information for the Standardization Rule in a project.

Request

Path Parameters

rule_name

Required *

string

The rule name or the asset id.

Query Parameters

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

location

Required *

string

The location of rule set. Required only when rule_name is not an asset id

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Create a new Standardization Rule.

POST /v3/quality_stage/rules/{rule_name}

create a new Standardization Rule in a given project.

Request

Path Parameters

rule_name

Required *

string

The rule name or the asset id.

Query Parameters

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

location

Required *

string

The location of rule set. Required only when rule_name is not an asset id

description

string

directory_asset_id

string

The directory asset id to create the asset in or move to

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Copy the Standardization Rule from from_location to to_location.

POST /v3/quality_stage/rules/{rule_name}/copy

Copy the Standardization Rule from from_location to to_location.

Request

Path Parameters

rule_name

Required *

string

The rule name or the asset id.

Query Parameters

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

from_location

Required *

string

The from_location.

to_location

string

The to_location. If not speecified, use Customized Standardization Rules/ + from_location

new_name

string

The new_name. the new name of the Standardization Rule. If not specified, use CopyOf + rule_name + number

description

string

directory_asset_id

string

The directory asset id to create the asset in or move to

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Test the Standardization Rule against a one-line test string (a single record) before you run it against an entire file.

POST /v3/quality_stage/rules/{rule_name}/test

Test the Standardize Rule against a one-line test string (a single record) before you run it against an entire file.

Request

Request Body

Json format for input string to be tested.

Path Parameters

rule_name

Required *

string

The rule name or the asset id.

Query Parameters

location

Required *

string

The location of rule set. Required only when rule_name is not an asset id

engine_type

string

The type of engine used for testing ruleset.

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Export the Standardization Rule in zip format.

GET /v3/quality_stage/rules/{rule_name}/zip

Export the Standardization Rule in zip format.

Request

Path Parameters

rule_name

Required *

string

The rule name or the asset id.

Query Parameters

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

location

Required *

string

The location of rule set. Required only when rule_name is not an asset id

force

boolean

Template rule only. Force download even without any changes, default is true

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Upload/update the Standardization Rule from the given zip file

PUT /v3/quality_stage/rules/{rule_name}/zip

Upload/update the Standardization Rule from the given zip file.

Request

Request Body

Path Parameters

rule_name

Required *

string

The rule name or the asset id.

Query Parameters

project_id

string

The ID of the project to use. `catalog_id` or `project_id` or `space_id` is required.

catalog_id

string

The ID of the catalog to use. `catalog_id` or `project_id` or `space_id` is required.

space_id

string

The ID of the space to use. `catalog_id` or `project_id` or `space_id` is required.

location

Required *

string

The location of rule set. Required only when rule_name is not an asset id

directory_asset_id

string

The directory asset id to create the asset in or move to

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

List DataStage XML schema libraries

GET /v3/schema_libraries

List existing DataStage XML schema libraries in the specified project, catalog, or space (either `project_id`, `catalog_id`, or `space_id` must be set).

Request

Path Parameters

Query Parameters

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

Create a new DataStage XML schema library

POST /v3/schema_libraries

Creates a new DataStage XML schema library in the specified project, catalog, or space (either `project_id`, `catalog_id`, or `space_id` must be set).

Request

Path Parameters

Query Parameters

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

directory_asset_id

string

The directory asset id to create the asset in or move to

name

Required *

string

The name of the new XML schema library.

folder

string

The folder that the new XML schema library belongs to.

description

string

The description of the new XML schema library.

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

Delete a DataStage XML schema library

DELETE /v3/schema_libraries/{library_id}

Delete a DataStage XML schema library in the specified project, catalog, or space (either `project_id`, `catalog_id`, or `space_id` must be set).

Request

Path Parameters

library_id

Required *

string

The ID of the XML Schema Library.

Query Parameters

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

Response

204 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

Get the specify DataStage XML schema library

GET /v3/schema_libraries/{library_id}

Get the specify DataStage XML schema library in the specified project, catalog, or space (either `project_id`, `catalog_id`, or `space_id` must be set).

Request

Path Parameters

library_id

Required *

string

The ID of the XML Schema Library.

Query Parameters

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

500 - An error occurred.

Update a DataStage XML schema library

POST /v3/schema_libraries/{library_id}

Update a DataStage XML schema library, name and directory_id only

Request

Path Parameters

library_id

Required *

string

The ID of the XML Schema Library.

Query Parameters

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

directory_asset_id

string

The directory asset id to create the asset in or move to

name

string

The new name of the XML schema library.

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

500 - An error occurred.

Upload a file to an existing DataStage XML schema library

PUT /v3/schema_libraries/{library_id}

Upload a file to an existing DataStage XML schema library in the specified project, catalog, or space (either `project_id`, `catalog_id`, or `space_id` must be set). Thread unsafe

Request

Request Body

The content of the file to upload.

Path Parameters

library_id

Required *

string

The ID of the XML Schema Library.

Query Parameters

file_name

string

The file name you want to upload to the specified XML schema library.

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

max_tree_size

integer

Max tree size for one type in schema library

output_global_type

boolean

Output the global type

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

Clone a DataStage XML schema library

POST /v3/schema_libraries/{library_id}/clone

Clone a DataStage XML schema library based on the specify library id in the specified project, catalog, or space (either `project_id`, `catalog_id`, or `space_id` must be set).

Request

Path Parameters

library_id

Required *

string

The ID of the XML Schema Library.

Query Parameters

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

directory_asset_id

string

The directory asset id to create the asset in or move to

name

string

The new name of the XML schema library.

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

500 - An error occurred.

Delete files from a DataStage XML schema library

DELETE /v3/schema_libraries/{library_id}/file

Delete files from a DataStage XML schema library based on the file_names in the specified project, catalog, or space (either `project_id`, `catalog_id`, or `space_id` must be set). Thread unsafe.

Request

Path Parameters

library_id

Required *

string

The ID of the XML Schema Library.

Query Parameters

file_names

Required *

string

The file names (path-dependent) you want to delete from the specified XML schema library. Multiple files can be specified by delimiting them with a comma. Skip files are not exist in this library.

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

max_tree_size

integer

Max tree size for one type in schema library

output_global_type

boolean

Output the global type

Response

204 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

Download file from a DataStage XML schema library

GET /v3/schema_libraries/{library_id}/file

Download file from a DataStage XML schema library based on the file_name in the specified project, catalog, or space (either `project_id`, `catalog_id`, or `space_id` must be set).

Request

Path Parameters

library_id

Required *

string

The ID of the XML Schema Library.

Query Parameters

file_name

string

The file name (path-dependent) you want to download from the specified XML schema library. If specified, only download the file. If not specified, download all files as a zip which maintain its original structure.

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

Rename a DataStage XML schema library

POST /v3/schema_libraries/{library_id}/rename

Rename a DataStage XML schema library based on the specify library id in the specified project, catalog, or space (either `project_id`, `catalog_id`, or `space_id` must be set).

Request

Path Parameters

library_id

Required *

string

The ID of the XML Schema Library.

Query Parameters

name

Required *

string

The new name of the XML schema library

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

500 - An error occurred.

Export a XML Schema Library in zip format

GET /v3/schema_libraries/{library_name}/zip

Export a XML Schema Library in zip format.

Request

Path Parameters

library_name

Required *

string

The name or id of the XML schema library

Query Parameters

folder

string

The folder of the XML schema library

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

Response

200 - Success.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

Import/create XML Schema Library from zip stream

PUT /v3/schema_libraries/{library_name}/zip

Import/create XML Schema Library from zip stream

Request

Request Body

Path Parameters

library_name

Required *

string

The name of the XML schema library

Query Parameters

folder

string

The folder of the XML schema library

conflict_option

string

The conflict_option. The default is skip

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

directory_asset_id

string

The directory asset id to create the asset in or move to

Response

200 - Success.

attachments

array

conflict_message

string

the message.

directory_asset

object

errors

array

folder

string

the folder.

library_id

string

map to asset id.

metadata

AssetSystemMetadata1

System metadata about an asset

name

string

parse_status

string

the status of the library, EMPTY/OK/ERROR/OUT_OF_SYNC.

skips

array

total_attachment

integer

total_error

integer

total_skip

integer

total_warning

integer

views

array

warnings

array

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

Delete table definitions

DELETE /v3/table_definitions

Delete the specified table definitions from a project or catalog (either project_id or catalog_id must be set).

Request

Path Parameters

Query Parameters

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

id

Required *

array

The list of table definitions IDs to delete.

Response

200 - The requested operation completed successfully.

202 - The requested operation is in progress.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An error occurred. See response for more information.

Field Match type Example asset.name Starts with ?asset.name=starts:MyTable asset.description Equals ?asset.description=starts:profiling

Field	Example
asset.name	?sort=+asset.name
metadata.create_time	?sort=-metadata.create_time

Multiple sort keys can be specified by delimiting them with a comma. For example, to sort in descending order on create_time and then in ascending order on name use: `?sort=-metadata.create_time,+asset.name ">

List table definitions

GET /v3/table_definitions

Lists the table definitions that are contained in the specified project or catalog (either project_id or catalog_id must be set).

Use the following parameters to filter the results:

Field	Match type	Example
asset.name	Starts with	?asset.name=starts:MyTable
asset.description	Equals	?asset.description=starts:profiling

Field	Example
asset.name	?sort=+asset.name
metadata.create_time	?sort=-metadata.create_time

Request

Path Parameters

Query Parameters

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

sort

string

The field to sort the results on, including whether to sort ascending (+) or descending (-), for example, sort=-metadata.create_time

start

string

The page token indicating where to start paging from.

limit

integer

The limit of the number of items to return, for example limit=50. If not specified a default of 100 will be used.

asset.name

string

Filter results based on the specified name.

asset.description

string

Filter results based on the specified description.

Response

200 - The requested operation completed successfully.

401 - You are not permitted to perform this action. See response for more information.

403 - Not authorized.

default - An error occurred. See response for more information.

Create table definition

POST /v3/table_definitions

Creates a table definition in the specified project or catalog (either project_id or catalog_id must be set). All subsequent calls to use the parameter set must specify the project or catalog ID the table definition was created in.

Request

Request Body

The table definition to be created.

Path Parameters

Query Parameters

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

directory_asset_id

string

The directory asset id to create the asset in or move to

asset_category

string

The category of the asset. Must be either SYSTEM or USER. Only a registered service can use this parameter.

Response

201 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - The data source type details cannot be found.

default - An error occurred. See response for more information.

Delete a table definition

DELETE /v3/table_definitions/{table_id}

Delete the specified table definition from a project or catalog (either project_id or catalog_id must be set).

Request

Path Parameters

table_id

Required *

string

Table definition ID

Query Parameters

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

Response

200 - The requested operation completed successfully.

400 - Bad request. See response for more information.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - The data source type details cannot be found.

410 - Gone. See response for more information.

default - An error occurred. See response for more information.

Get table definition

GET /v3/table_definitions/{table_id}

Get table definition

Request

Path Parameters

table_id

Required *

string

Table definition ID

Query Parameters

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

Response

200 - Table definition found.

400 - Bad request. See response for more information.

401 - You are not authorized to retrieve the table definition.

403 - You are not permitted to perform this action.

404 - The data source type details cannot be found.

429 - The service is currently receiving more requests than it can process in a timely fashion. Please retry submitting your request later.

500 - An error occurred. No table definitions were retrieved.

504 - A timeout occurred when processing your request. Please retry later.

Patch a table definition

PATCH /v3/table_definitions/{table_id}

Patch a table definition in the specified project or catalog (either project_id or catalog_id must be set).

Request

Request Body

The patch operations to apply.

Path Parameters

table_id

Required *

string

Table definition ID

Query Parameters

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

Response

201 - The requested operation completed successfully.

400 - Bad request. See response for more information.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - The data source type details cannot be found.

default - An error occurred. See response for more information.

Update a table definition with a replacement

PUT /v3/table_definitions/{table_id}

Replace the contents of a table definition in the specified project or catalog (either project_id or catalog_id must be set).

Request

Request Body

The table definition to be updated.

Path Parameters

table_id

Required *

string

Table definition ID

Query Parameters

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

directory_asset_id

string

The directory asset id to create the asset in or move to

Response

201 - The requested operation completed successfully.

400 - Bad request. See response for more information.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - The data source type details cannot be found.

default - An error occurred. See response for more information.

Clone table definition

POST /v3/table_definitions/{table_id}/clone

Clone table definition

Request

Path Parameters

table_id

Required *

string

Table definition ID

Query Parameters

catalog_id

string

The ID of the catalog to use. catalog_id, space_id, or project_id is required.

project_id

string

The ID of the project to use. catalog_id, space_id, or project_id is required.

space_id

string

The ID of the space to use. catalog_id, space_id, or project_id is required.

directory_asset_id

string

The directory asset id to create the asset in or move to

name

string

The new name.

Delete assets and their attachments

Request

Path Parameters

Query Parameters

project_id

space_id

asset_ids

asset_type

purge_test_data

Response

200 - The requested operation completed successfully.

202 - The requested operation is in progress.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

List assets

Request

Path Parameters

Query Parameters

project_id

space_id

asset_type

sort

start

limit

asset.name

asset.description

asset.resource_key

tags

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

Create asset

Request

Request Body

Path Parameters

Query Parameters

project_id

space_id

directory_asset_id

Response

201 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - The data source type details cannot be found.

default - An unexpected error occurred. See response for more information.

Create/update asset based on the given zip file.

Request

Request Body

Path Parameters

Query Parameters

project_id

space_id

asset_type

directory_asset_id

override

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

404 - The data source type details cannot be found.

default - An unexpected error occurred. See response for more information.

Analyze a Flow for Unit Testing

Request

Path Parameters

flow_id

Query Parameters

project_id

space_id

Response

200 - The requested operation completed successfully.

401 - You are not authorized to access the service. See response for more information.

403 - You are not permitted to perform this action. See response for more information.

default - An unexpected error occurred. See response for more information.

Create Test Case specification from Flow Analysis

Request

Request Body

analysis