Infer text

POST /ml/v1/text/generation

Infer the next tokens for a given deployed model with a set of parameters.

This API is legacy, consider using Text Chat.

Auditing

Calling this method generates the following auditing event.

• pm-20.foundation-model.send

Request

Request Body

TextGenRequest

From a given prompt, infer the next tokens.

input

Required *

string

The prompt to generate completions. Note: The method tokenizes the input internally. It is recommended not to leave any trailing spaces.

model_id

Required *

string

The `id` of the model to be used for this request. Please refer to the list of models.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

parameters

TextGenParameters

Properties that control the model and response.

decoding_method

string

Represents the strategy used for picking the tokens during generation of the output text.

During text generation when parameter value is set to greedy, each successive token corresponds to the highest probability token given the text that has already been generated. This strategy can lead to repetitive results especially for longer output sequences. The alternative sample strategy generates text by picking subsequent tokens based on the probability distribution of possible next tokens defined by (i.e., conditioned on) the already-generated text and the top_k and top_p parameters described below. See this url for an informative article about text generation.

In future implementations, this parameter will be automatically set depending on the `temperature` parameter so this parameter will be ignored.

length_penalty

object

It can be used to exponentially increase the likelihood of the text generation terminating once a specified number of tokens have been generated.

In future implementations, this parameter will be ignored.

max_new_tokens

integer

The maximum number of new tokens to be generated. The maximum supported value for this field depends on the model being used.

How the "token" is defined depends on the tokenizer and vocabulary size, which in turn depends on the model. Often the tokens are a mix of full words and sub-words. To learn more about tokenization, see here.

Depending on the users plan, and on the model being used, there may be an enforced maximum number of new tokens.

min_new_tokens

integer

If stop sequences are given, they are ignored until minimum tokens are generated.

random_seed

integer

Random number generator seed to use in sampling mode for experimental repeatability.

stop_sequences

array

Stop sequences are one or more strings which will cause the text generation to stop if/when they are produced as part of the output. Stop sequences encountered prior to the minimum number of tokens being generated will be ignored.

temperature

number

A value used to modify the next-token probabilities in sampling mode. Values less than 1.0 sharpen the probability distribution, resulting in "less random" output. Values greater than 1.0 flatten the probability distribution, resulting in "more random" output. A value of 1.0 has no effect.

time_limit

integer

Time limit in milliseconds - if not completed within this time, generation will stop. The text generated so far will be returned along with the TIME_LIMIT stop reason.

Depending on the users plan, and on the model being used, there may be an enforced maximum time limit.

top_k

integer

The number of highest probability vocabulary tokens to keep for top-k-filtering. Only applies for sampling mode. When decoding_strategy is set to sample, only the top_k most likely tokens are considered as candidates for the next generated token.

top_p

number

Similar to top_k except the candidates to generate the next token are the most likely tokens with probabilities that add up to at least top_p. Also known as nucleus sampling. A value of 1.0 is equivalent to disabled.

repetition_penalty

number

Represents the penalty for penalizing tokens that have already been generated or belong to the context. The value 1.0 means that there is no penalty.

truncate_input_tokens

integer

Represents the maximum number of input tokens accepted. This can be used to avoid requests failing due to input being longer than configured limits. If the text is truncated, then it truncates the start of the input (on the left), so the end of the input will remain the same. If this value exceeds the `maximum sequence length` (refer to the documentation to find this value for the model) then the call will fail if the total number of tokens exceeds the `maximum sequence length`.

return_options

object

Properties that control what is returned.

include_stop_sequence

boolean

Pass `false` to omit matched stop sequences from the end of the output text. The default is `true`, meaning that the output will end with the stop sequence text when matched.

moderations

Moderations

Properties that control the moderations, for usages such as `Hate and profanity` (HAP) and `Personal identifiable information` (PII) filtering. This list can be extended with new types of moderations.

hap

object

The properties for the moderation. Each type of moderation may have additional properties that are specific to that moderation.

pii

object

The properties for the moderation. Each type of moderation may have additional properties that are specific to that moderation.

granite_guardian

object

The properties specific to Granite Guardian. This detector is in beta and may change.

input_ranges

array

If set, then only these ranges will be applied to the moderations. This is useful in the case that certain parts of the input text have already been checked.

crypto

CryptoInfo

To enable encryption, configure credentials for your chosen keys management service. For details, see Encrypting inference requests.

key_ref

Required *

string

Identifier of the DEK in the chosen keys management service.

• IBM Key Protect - full CRN (e.g. `crn:v1:bluemix:public:kms:us-south:a/12345:<instance_id>:key:<root_key_id>:wdek:<ciphertext>`) This field is required, but its exact format depends on the selected `keys_manager`. No strict pattern is enforced here - validation will be performed by the keys management service.

Response

200 - Successful operation

model_id

string

The `id` of the model for inference.

model_version

string

The model version (using semantic versioning) if set.

created_at

string

The time when the response was created in ISO 8601 format.

results

array

The generated tokens.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Infer text event stream

POST /ml/v1/text/generation_stream

Infer the next tokens for a given deployed model with a set of parameters. This operation will return the output tokens as a stream of events.

This API is legacy, consider using Text Chat Stream.

Auditing

Calling this method generates the following auditing event.

• pm-20.foundation-model.send

Request

Request Body

TextGenStreamRequest

From a given prompt, infer the next tokens in a server-sent events (SSE) stream.

input

Required *

string

The prompt to generate completions. Note: The method tokenizes the input internally. It is recommended not to leave any trailing spaces.

model_id

Required *

string

The `id` of the model to be used for this request. Please refer to the list of models.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

parameters

TextGenParameters

Properties that control the model and response.

decoding_method

string

Represents the strategy used for picking the tokens during generation of the output text.

During text generation when parameter value is set to greedy, each successive token corresponds to the highest probability token given the text that has already been generated. This strategy can lead to repetitive results especially for longer output sequences. The alternative sample strategy generates text by picking subsequent tokens based on the probability distribution of possible next tokens defined by (i.e., conditioned on) the already-generated text and the top_k and top_p parameters described below. See this url for an informative article about text generation.

In future implementations, this parameter will be automatically set depending on the `temperature` parameter so this parameter will be ignored.

length_penalty

object

It can be used to exponentially increase the likelihood of the text generation terminating once a specified number of tokens have been generated.

In future implementations, this parameter will be ignored.

max_new_tokens

integer

The maximum number of new tokens to be generated. The maximum supported value for this field depends on the model being used.

How the "token" is defined depends on the tokenizer and vocabulary size, which in turn depends on the model. Often the tokens are a mix of full words and sub-words. To learn more about tokenization, see here.

Depending on the users plan, and on the model being used, there may be an enforced maximum number of new tokens.

min_new_tokens

integer

If stop sequences are given, they are ignored until minimum tokens are generated.

random_seed

integer

Random number generator seed to use in sampling mode for experimental repeatability.

stop_sequences

array

Stop sequences are one or more strings which will cause the text generation to stop if/when they are produced as part of the output. Stop sequences encountered prior to the minimum number of tokens being generated will be ignored.

temperature

number

A value used to modify the next-token probabilities in sampling mode. Values less than 1.0 sharpen the probability distribution, resulting in "less random" output. Values greater than 1.0 flatten the probability distribution, resulting in "more random" output. A value of 1.0 has no effect.

time_limit

integer

Time limit in milliseconds - if not completed within this time, generation will stop. The text generated so far will be returned along with the TIME_LIMIT stop reason.

Depending on the users plan, and on the model being used, there may be an enforced maximum time limit.

top_k

integer

The number of highest probability vocabulary tokens to keep for top-k-filtering. Only applies for sampling mode. When decoding_strategy is set to sample, only the top_k most likely tokens are considered as candidates for the next generated token.

top_p

number

Similar to top_k except the candidates to generate the next token are the most likely tokens with probabilities that add up to at least top_p. Also known as nucleus sampling. A value of 1.0 is equivalent to disabled.

repetition_penalty

number

Represents the penalty for penalizing tokens that have already been generated or belong to the context. The value 1.0 means that there is no penalty.

truncate_input_tokens

integer

Represents the maximum number of input tokens accepted. This can be used to avoid requests failing due to input being longer than configured limits. If the text is truncated, then it truncates the start of the input (on the left), so the end of the input will remain the same. If this value exceeds the `maximum sequence length` (refer to the documentation to find this value for the model) then the call will fail if the total number of tokens exceeds the `maximum sequence length`.

return_options

object

Properties that control what is returned.

include_stop_sequence

boolean

Pass `false` to omit matched stop sequences from the end of the output text. The default is `true`, meaning that the output will end with the stop sequence text when matched.

moderations

Moderations

Properties that control the moderations, for usages such as `Hate and profanity` (HAP) and `Personal identifiable information` (PII) filtering. This list can be extended with new types of moderations.

hap

object

The properties for the moderation. Each type of moderation may have additional properties that are specific to that moderation.

pii

object

The properties for the moderation. Each type of moderation may have additional properties that are specific to that moderation.

granite_guardian

object

The properties specific to Granite Guardian. This detector is in beta and may change.

input_ranges

array

If set, then only these ranges will be applied to the moderations. This is useful in the case that certain parts of the input text have already been checked.

Response

200 - Successful operation (`Content-Type: text/event-stream`).

400 -

401 -

403 -

404 -

Text tokenization

POST /ml/v1/text/tokenization

The text tokenize operation allows you to check the conversion of provided input to tokens for a given model. It splits text into words or sub-words, which then are converted to ids through a look-up table (vocabulary). Tokenization allows the model to have a reasonable vocabulary size.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-tokenization.send

Request

Request Body

TextTokenizeRequest

The input string to tokenize.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

model_id

Required *

string

The `id` of the model to be used for this request. Please refer to the list of models.

input

Required *

string

The input string to tokenize.

parameters

TextTokenizeParameters

The parameters for text tokenization.

return_tokens

boolean

If this is `true` then the actual tokens will also be returned in the response.

crypto

CryptoInfo

To enable encryption, configure credentials for your chosen keys management service. For details, see Encrypting inference requests.

key_ref

Required *

string

Identifier of the DEK in the chosen keys management service.

• IBM Key Protect - full CRN (e.g. `crn:v1:bluemix:public:kms:us-south:a/12345:<instance_id>:key:<root_key_id>:wdek:<ciphertext>`) This field is required, but its exact format depends on the selected `keys_manager`. No strict pattern is enforced here - validation will be performed by the keys management service.

Response

200 - Successful operation

model_id

string

The `id` of the model to be used for this request. Please refer to the list of models.

result

TextTokenizeResult

The result of tokenizing the input string.

400 -

401 -

403 -

404 -

List the available foundation models

GET /ml/v1/foundation_model_specs

Retrieve the list of deployed foundation models.

Request

Response

200 - OK

total_count

integer

The total number of resources.

limit

integer

The number of items to return in each page.

first

PaginationFirst

The reference to the first item in the current page.

next

PaginationNext

A reference to the first item of the next page, if any.

resources

FoundationModelsArray

The supported foundation models.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

404 -

List the supported tasks

GET /ml/v1/foundation_model_tasks

Retrieve the list of tasks that are supported by the foundation models.

Request

Response

200 - OK

total_count

integer

The total number of resources.

limit

integer

The number of items to return in each page.

first

PaginationFirst

The reference to the first item in the current page.

next

PaginationNext

A reference to the first item of the next page, if any.

resources

FoundationModelTasksArray

The supported foundation model tasks.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

404 -

POST /ml/v4/deployments

Create a new deployment, currently the only supported type is `online`.

If this is a deployment for a prompt template then the `prompt_template` object should exist and the `id` must be the `id` of the prompt template to be deployed.

Auditing

Calling this method generates the following auditing event.

• pm-20.deployment.create

Request

Request Body

DeploymentResourcePrototype

The deployment request entity.

The following important fields are described for each use case:

1. Prompt template:
   • `base_model_id`: required
   • `prompt_template.id`: required
   • `online`: required
   • `hardware_spec`: forbidden
   • `hardware_request`: forbidden
   • response `deployed_asset_type`: `foundation_model`
2. Custom foundation model:
   • `asset.id`: required
   • `online`: required
   • `online.parameters.foundation_model`: optional
   • `hardware_spec`: required
   • `hardware_request`: forbidden
   • `base_model_id`: forbidden
   • `base_deployment_id`: forbidden
   • response `deployed_asset_type`: `custom_foundation_model`
3. Deploy on Demand model:
   • `asset.id`: required
   • `online`: required
   • `online.parameters.foundation_model`: forbidden
   • `hardware_spec`: optional
   • `hardware_request`: forbidden
   • `base_model_id`: forbidden
   • `base_deployment_id`: forbidden
   • `space_id`: required
   • `project_id`: forbidden
   • response `deployed_asset_type`: `curated_foundation_model`

name

Required *

string

The name of the resource.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

description

string

A description of the resource.

tags

Tags

A list of tags for this resource.

custom

Custom

User defined properties specified as key-value pairs.

prompt_template

SimpleRel

A reference to a resource.

id

Required *

string

The id of the referenced resource.

online

Required *

OnlineDeployment

Indicates that this is an online deployment. An object has to be specified but can be empty. The `serving_name` can be provided in the `online.parameters`.

parameters

object

A set of key-value pairs that are used to configure the deployment.

hardware_spec

HardwareSpec

A hardware specification.

id

string

The id of the hardware specification.

rev

string

The revision of the hardware specification.

name

string

The name of the hardware specification.

num_nodes

integer

The number of nodes applied to a computation.

hardware_request

HardwareRequest

The requested hardware for deployment.

size

string

The size of GPU requested for the deployment.

num_nodes

number

The number of nodes for the GPU requested for deployment.

asset

Rel

A reference to a resource.

id

Required *

string

The id of the referenced resource.

rev

string

The revision of the referenced resource.

base_model_id

string

The base model that is required for this deployment if this is for a prompt template for an IBM foundation model.

Response

202 - Deployment created.

metadata

ResourceMeta

Common metadata for a simple resource.

entity

DeploymentEntity

The common fields between a request and the response.

400 -

401 -

403 -

404 -

Retrieve the deployments

GET /ml/v4/deployments

Retrieve the list of deployments for the specified space or project.

Auditing

Calling this method generates the following auditing event.

• pm-20.deployment.list

Request

Path Parameters

Query Parameters

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

serving_name

string

Retrieves the deployment, if any, that contains this `serving_name`.

tag.value

string

Retrieves only the resources with the given tag value.

asset_id

string

Retrieves only the resources with the given asset_id, asset_id would be the model id.

prompt_template_id

string

Retrieves only the resources with the given prompt_template_id.

name

string

Retrieves only the resources with the given name.

type

string

Retrieves the resources filtered with the given type. There are the deployment types as well as an additional `prompt_template` if the deployment type includes a prompt template.

The supported deployment types are (see the description for `deployed_asset_type` in the deployment entity):

1. `foundation_model` - when a prompt template is used on a pre-deployed IBM provided model.

2. `custom_foundation_model` - when a custom foundation model is deployed.

3. `curated_foundation_model` - when a deploy on demand model is deployed. These can be combined with the flag `prompt_template` like this:

4. `type=foundation_model` - return all prompt template deployments.

5. `type=foundation_model and prompt_template` - return all prompt template deployments - this is the same as the previous query because a `foundation_model` can only exist with a prompt template.

6. `type=prompt_template` - return all deployments with a prompt template.

state

string

Retrieves the resources filtered by state. Allowed values are `initializing`, `updating`, `ready` and `failed`.

conflict

boolean

Returns whether `serving_name` is available for use or not. This query parameter cannot be combined with any other parameter except for `serving_name`.

Response

200 - OK.

total_count

integer

The total number of resources. Computed explicitly only when `total_count=true` query parameter is present. This is in order to avoid performance penalties.

limit

integer

The number of items to return in each page.

first

PaginationFirst

The reference to the first item in the current page.

next

PaginationNext

A reference to the first item of the next page, if any.

resources

array

A list of deployment resources.

system

DeploymentSystem

System details including warnings.

204 - `serving_name` is available for use.

Returned when `serving_name` and `conflict` query parameters are used.

400 -

401 -

403 -

404 -

409 - Returned when `serving_name` and `conflict` query parameters are used.

The response body will contain the reason.

trace

string

An identifier that can be used to trace the request.

status_code

integer

The corresponding error code in integer.

errors

array

The list of errors.

Retrieve the deployment details

GET /ml/v4/deployments/{deployment_id}

Retrieve the deployment details with the specified identifier.

Auditing

Calling this method generates the following auditing event.

• pm-20.deployment.read

Request

Path Parameters

Query Parameters

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

attempt_activation

boolean

Attempts activation for the deployment with specified `deployment_id`, if it is hibernated and if it is set to true.

Response

200 - Deployment details.

metadata

ResourceMeta

Common metadata for a simple resource.

entity

DeploymentEntity

The common fields between a request and the response.

400 -

401 -

403 -

404 -

PATCH /ml/v4/deployments/{deployment_id}

Update the deployment metadata. The following parameters of deployment metadata are supported for the patch operation.

• `/name`
• `/description`
• `/tags`
• `/custom`
• `/online/parameters`
• `/asset` - `replace` only
• `/prompt_template` - `replace` only
• `/hardware_spec`
• `/hardware_request`
• `/base_model_id` - `replace` only (applicable only to prompt template deployments referring to IBM base foundation models)

The PATCH operation with path specified as `/online/parameters` can be used to update the `serving_name`.

Auditing

Calling this method generates the following auditing event.

• pm-20.deployment.update

Request

Request Body

The json patch.

Path Parameters

Query Parameters

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

Response

202 - Deployment accepted

metadata

ResourceMeta

Common metadata for a simple resource.

entity

DeploymentEntity

The common fields between a request and the response.

400 -

401 -

403 -

404 -

Delete the deployment

DELETE /ml/v4/deployments/{deployment_id}

Delete the deployment with the specified identifier.

Auditing

Calling this method generates the following auditing event.

• pm-20.deployment.delete

Request

Path Parameters

Query Parameters

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

Response

204 - Deployment deleted.

400 -

401 -

404 -

Return options

Note that there is currently a limitation in this operation when using `return_options`, for input only `input_text` will be returned if requested, for output the `input_tokens` and `generated_tokens` will not be returned. ">

Infer text

POST /ml/v1/deployments/{id_or_name}/text/generation

Infer the next tokens for a given deployed model with a set of parameters. If a `serving_name` is used then it must match the `serving_name` that is returned in the `inference` section when the deployment was created.

This API is legacy, consider using Deployment Text Chat.

Return options

Note that there is currently a limitation in this operation when using `return_options`, for input only `input_text` will be returned if requested, for output the `input_tokens` and `generated_tokens` will not be returned.

Auditing

Calling this method generates the following auditing event.

• pm-20.foundation-model.send

Request

Request Body

DeploymentTextGenRequest

From a given prompt, infer the next tokens.

input

string

The prompt to generate completions. Note: The method tokenizes the input internally. It is recommended not to leave any trailing spaces.

This field is ignored if there is a prompt template.

parameters

DeploymentTextGenProperties

Properties that control the model and response.

decoding_method

string

Represents the strategy used for picking the tokens during generation of the output text.

During text generation when parameter value is set to greedy, each successive token corresponds to the highest probability token given the text that has already been generated. This strategy can lead to repetitive results especially for longer output sequences. The alternative sample strategy generates text by picking subsequent tokens based on the probability distribution of possible next tokens defined by (i.e., conditioned on) the already-generated text and the top_k and top_p parameters described below. See this url for an informative article about text generation.

In future implementations, this parameter will be automatically set depending on the `temperature` parameter so this parameter will be ignored.

length_penalty

object

It can be used to exponentially increase the likelihood of the text generation terminating once a specified number of tokens have been generated.

In future implementations, this parameter will be ignored.

max_new_tokens

integer

The maximum number of new tokens to be generated. The maximum supported value for this field depends on the model being used.

How the "token" is defined depends on the tokenizer and vocabulary size, which in turn depends on the model. Often the tokens are a mix of full words and sub-words. To learn more about tokenization, see here.

Depending on the users plan, and on the model being used, there may be an enforced maximum number of new tokens.

min_new_tokens

integer

If stop sequences are given, they are ignored until minimum tokens are generated.

random_seed

integer

Random number generator seed to use in sampling mode for experimental repeatability.

stop_sequences

array

Stop sequences are one or more strings which will cause the text generation to stop if/when they are produced as part of the output. Stop sequences encountered prior to the minimum number of tokens being generated will be ignored.

temperature

number

A value used to modify the next-token probabilities in sampling mode. Values less than 1.0 sharpen the probability distribution, resulting in "less random" output. Values greater than 1.0 flatten the probability distribution, resulting in "more random" output. A value of 1.0 has no effect.

time_limit

integer

Time limit in milliseconds - if not completed within this time, generation will stop. The text generated so far will be returned along with the TIME_LIMIT stop reason.

Depending on the users plan, and on the model being used, there may be an enforced maximum time limit.

top_k

integer

The number of highest probability vocabulary tokens to keep for top-k-filtering. Only applies for sampling mode. When decoding_strategy is set to sample, only the top_k most likely tokens are considered as candidates for the next generated token.

top_p

number

Similar to top_k except the candidates to generate the next token are the most likely tokens with probabilities that add up to at least top_p. Also known as nucleus sampling. A value of 1.0 is equivalent to disabled.

repetition_penalty

number

Represents the penalty for penalizing tokens that have already been generated or belong to the context. The value 1.0 means that there is no penalty.

truncate_input_tokens

integer

Represents the maximum number of input tokens accepted. This can be used to avoid requests failing due to input being longer than configured limits. If the text is truncated, then it truncates the start of the input (on the left), so the end of the input will remain the same. If this value exceeds the `maximum sequence length` (refer to the documentation to find this value for the model) then the call will fail if the total number of tokens exceeds the `maximum sequence length`.

return_options

object

Properties that control what is returned.

include_stop_sequence

boolean

Pass `false` to omit matched stop sequences from the end of the output text. The default is `true`, meaning that the output will end with the stop sequence text when matched.

typical_p

number

Local typicality measures how similar the conditional probability of predicting a target token next is to the expected conditional probability of predicting a random token next, given the partial text already generated. If less than 1, the smallest set of the most locally typical tokens with probabilities that add up to typical_p or higher are kept for generation.

prompt_variables

object

The prompt variables.

moderations

Moderations

Properties that control the moderations, for usages such as `Hate and profanity` (HAP) and `Personal identifiable information` (PII) filtering. This list can be extended with new types of moderations.

hap

object

The properties for the moderation. Each type of moderation may have additional properties that are specific to that moderation.

pii

object

The properties for the moderation. Each type of moderation may have additional properties that are specific to that moderation.

granite_guardian

object

The properties specific to Granite Guardian. This detector is in beta and may change.

input_ranges

array

If set, then only these ranges will be applied to the moderations. This is useful in the case that certain parts of the input text have already been checked.

Response

200 - Successful operation

model_id

string

The `id` of the model for inference.

model_version

string

The model version (using semantic versioning) if set.

created_at

string

The time when the response was created in ISO 8601 format.

results

array

The generated tokens.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Return options

Note that there is currently a limitation in this operation when using `return_options`, for input only `input_text` will be returned if requested, for output the `input_tokens` and `generated_tokens` will not be returned, also the `rank` and `top_tokens` will not be returned. ">

Infer text event stream

POST /ml/v1/deployments/{id_or_name}/text/generation_stream

Infer the next tokens for a given deployed model with a set of parameters. This operation will return the output tokens as a stream of events. If a `serving_name` is used then it must match the `serving_name` that is returned in the `inference` section when the deployment was created.

This API is legacy, consider using Deployment Text Chat Stream.

Return options

Note that there is currently a limitation in this operation when using `return_options`, for input only `input_text` will be returned if requested, for output the `input_tokens` and `generated_tokens` will not be returned, also the `rank` and `top_tokens` will not be returned.

Auditing

Calling this method generates the following auditing event.

• pm-20.foundation-model.send

Request

Request Body

DeploymentTextGenRequest

From a given prompt, infer the next tokens in a server-sent events (SSE) stream.

input

string

The prompt to generate completions. Note: The method tokenizes the input internally. It is recommended not to leave any trailing spaces.

This field is ignored if there is a prompt template.

parameters

DeploymentTextGenProperties

Properties that control the model and response.

decoding_method

string

Represents the strategy used for picking the tokens during generation of the output text.

During text generation when parameter value is set to greedy, each successive token corresponds to the highest probability token given the text that has already been generated. This strategy can lead to repetitive results especially for longer output sequences. The alternative sample strategy generates text by picking subsequent tokens based on the probability distribution of possible next tokens defined by (i.e., conditioned on) the already-generated text and the top_k and top_p parameters described below. See this url for an informative article about text generation.

In future implementations, this parameter will be automatically set depending on the `temperature` parameter so this parameter will be ignored.

length_penalty

object

It can be used to exponentially increase the likelihood of the text generation terminating once a specified number of tokens have been generated.

In future implementations, this parameter will be ignored.

max_new_tokens

integer

The maximum number of new tokens to be generated. The maximum supported value for this field depends on the model being used.

How the "token" is defined depends on the tokenizer and vocabulary size, which in turn depends on the model. Often the tokens are a mix of full words and sub-words. To learn more about tokenization, see here.

Depending on the users plan, and on the model being used, there may be an enforced maximum number of new tokens.

min_new_tokens

integer

If stop sequences are given, they are ignored until minimum tokens are generated.

random_seed

integer

Random number generator seed to use in sampling mode for experimental repeatability.

stop_sequences

array

Stop sequences are one or more strings which will cause the text generation to stop if/when they are produced as part of the output. Stop sequences encountered prior to the minimum number of tokens being generated will be ignored.

temperature

number

A value used to modify the next-token probabilities in sampling mode. Values less than 1.0 sharpen the probability distribution, resulting in "less random" output. Values greater than 1.0 flatten the probability distribution, resulting in "more random" output. A value of 1.0 has no effect.

time_limit

integer

Time limit in milliseconds - if not completed within this time, generation will stop. The text generated so far will be returned along with the TIME_LIMIT stop reason.

Depending on the users plan, and on the model being used, there may be an enforced maximum time limit.

top_k

integer

The number of highest probability vocabulary tokens to keep for top-k-filtering. Only applies for sampling mode. When decoding_strategy is set to sample, only the top_k most likely tokens are considered as candidates for the next generated token.

top_p

number

Similar to top_k except the candidates to generate the next token are the most likely tokens with probabilities that add up to at least top_p. Also known as nucleus sampling. A value of 1.0 is equivalent to disabled.

repetition_penalty

number

Represents the penalty for penalizing tokens that have already been generated or belong to the context. The value 1.0 means that there is no penalty.

truncate_input_tokens

integer

Represents the maximum number of input tokens accepted. This can be used to avoid requests failing due to input being longer than configured limits. If the text is truncated, then it truncates the start of the input (on the left), so the end of the input will remain the same. If this value exceeds the `maximum sequence length` (refer to the documentation to find this value for the model) then the call will fail if the total number of tokens exceeds the `maximum sequence length`.

return_options

object

Properties that control what is returned.

include_stop_sequence

boolean

Pass `false` to omit matched stop sequences from the end of the output text. The default is `true`, meaning that the output will end with the stop sequence text when matched.

typical_p

number

Local typicality measures how similar the conditional probability of predicting a target token next is to the expected conditional probability of predicting a random token next, given the partial text already generated. If less than 1, the smallest set of the most locally typical tokens with probabilities that add up to typical_p or higher are kept for generation.

prompt_variables

object

The prompt variables.

moderations

Moderations

Properties that control the moderations, for usages such as `Hate and profanity` (HAP) and `Personal identifiable information` (PII) filtering. This list can be extended with new types of moderations.

hap

object

The properties for the moderation. Each type of moderation may have additional properties that are specific to that moderation.

pii

object

The properties for the moderation. Each type of moderation may have additional properties that are specific to that moderation.

granite_guardian

object

The properties specific to Granite Guardian. This detector is in beta and may change.

input_ranges

array

If set, then only these ranges will be applied to the moderations. This is useful in the case that certain parts of the input text have already been checked.

Response

200 - Successful operation (`Content-Type: text/event-stream`).

400 -

401 -

403 -

404 -

If `stream` is true, this operation will return the output tokens in a server-sent events (SSE) stream. ">

Infer chat completions

POST /ml/v1/deployments/{id_or_name}/chat/completions

Infers the next chat message for a given deployment. The deployment must reference either a prompt template with `input_mode` set to `chat`, a custom foundation model, or a curated foundation model. When a prompt template is referenced, the model used for the chat request is specified by the deployment's `base_model_id`. Chat parameters are derived from the prompt template's `model_parameters`. If a `serving_name` is provided, it must match the `serving_name` returned in the inference section at the time of deployment creation.

Related guides: - Deployment - Prompt template - Text chat

If `stream` is true, this operation will return the output tokens in a server-sent events (SSE) stream.

Auditing

Calling this method generates the following auditing event.

• chat-completions.send

Request

Request Body

DeploymentTextChatRequest

From a given prompt, infer the next chat message.

messages

Required *

array

The messages for this chat session.

If the deployment references a prompt template then `system` `role` can not be in `messages`. For such deployments, depending on the model, the `content` of `system` `role` may be from `system_prompt` of the prompt template, and will be automatically inserted into `messages`. As an example, depending on the model, if `system_prompt` of a prompt template is "You are Granite Chat, an AI language model developed by IBM. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.", a message with `system` `role` having `content` the same as `system_prompt` is inserted.

tools

array

Tool functions that can be called with the response.

tool_choice_option

string

Specify either `tool_choice_option` to allow the model to pick or `tool_choice` to force the model to call a tool.

Using `auto` means the model can pick between generating a message or calling one or more tools. Default is `auto`.

Using `none` means the model will not call any tool and instead generates a message.

Using `required` means the model must call one or more tools.

tool_choice

TextChatToolChoiceTool

Specifying a particular tool via `{"type": "function", "function": {"name": "my_function"}}` forces the model to call that tool. Specify either `tool_choice_option` to allow the model to pick or `tool_choice` to force the model to call a tool.

type

Required *

string

The tool type.

function

Required *

object

The named function.

context

string

If specified, `context` will be inserted into `messages`. Depending on the model, `context` may be inserted into the `content` with `system` `role`; or into the `content` of the last message of `user` `role`.

In the example, `context` "Today is Wednesday" is inserted as such `content` of `user` becomes "Today is Wednesday. Who are you and which day is tomorrow?"

chat_template_kwargs

object

Additional kwargs to pass to the chat template, described as a JSON Schema object. See the JSON Schema reference for documentation about the format.

frequency_penalty

number

Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

include_reasoning

boolean

Whether to include `reasoning_content` in the response. Default is `true`.

logit_bias

object

Increasing or decreasing probability of tokens being selected during generation; a positive bias makes a token more likely to appear, while a negative bias makes it less likely.

logprobs

boolean

Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.

top_logprobs

integer

An integer specifying the number of most likely tokens to return at each token position, each with an associated log probability. The option `logprobs` must be set to `true` if this parameter is used.

max_completion_tokens

integer

The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. Set to 0 for the model's configured max generated tokens.

max_tokens

integer

The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. Set to 0 for the model's configured max generated tokens.

This value is now deprecated in favor of `max_completion_tokens`. If specified together with `max_completion_tokens`, `max_tokens` will be ignored.

n

integer

How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Keep n as 1 to minimize costs.

presence_penalty

number

Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

reasoning_effort

string

A lower reasoning effort can result in faster responses, fewer tokens used, and shorter `reasoning_content` in the responses. Supported values are `low`, `medium`, and `high`.

response_format

TextChatResponseFormat

The chat response format parameters.

type

Required *

string

Used to enable JSON mode, which guarantees the message the model generates is valid JSON.

Important: when using JSON mode, you must also instruct the model to produce JSON yourself via a system or user message. Without this, the model may generate an unending stream of whitespace until the generation reaches the token limit, resulting in a long-running and seemingly "stuck" request. Also note that the message content may be partially cut off if `finish_reason="length"`, which indicates the generation exceeded `max_tokens` or the conversation exceeded the max context length.

• `text` : Will not create a JSON structured output.
• `json_object` : User will get a JSON structured output, but the schema is not guranteed.
• `json_schema` : User needs to declare the `json_schema` object to provide the desired JSON schema in order to get a matched response.

json_schema

object

User-defined JSON schema object with optional parameters to enforce structured output.

Important: this field will only applied when `response_format`.`type` is set to `json_schema`.

seed

integer

Random number generator seed to use in sampling mode for experimental repeatability.

stop

array

Stop sequences are one or more strings which will cause the text generation to stop if/when they are produced as part of the output. Stop sequences encountered prior to the minimum number of tokens being generated will be ignored.

stream

boolean

If set to true, this operation will return the output tokens as a stream of events, and usage is always included in the response.

temperature

number

What sampling temperature to use,. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.

We generally recommend altering this or `top_p` but not both.

top_p

number

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

We generally recommend altering this or `temperature` but not both.

time_limit

integer

Time limit in milliseconds - if not completed within this time, generation will stop. The text generated so far will be returned along with the `TIME_LIMIT`` stop reason. Depending on the users plan, and on the model being used, there may be an enforced maximum time limit.

Response

200 - Successful operation. `Content-Type: text/event-stream` if `stream` is true.

id

string

A unique identifier for the chat completion.

model_id

string

The model used for the chat completion.

model

string

This field is a duplicate of `model_id` and is provided in order to provide better compatibility with other APIs.

model_version

string

The model version (using semantic versioning) if set.

created

integer

The Unix timestamp (in seconds) of when the chat completion was created.

created_at

string

The time when the response was created in ISO 8601 format.

usage

TextChatUsage

Usage statistics for the completion request.

moderations

ModerationResults

The result of any detected moderations.

choices

array

A list of chat completion choices. Can be more than one if `n` is greater than 1.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

You can also use Deployment Chat Completions to achieve the same result. ">

Infer text chat

POST /ml/v1/deployments/{id_or_name}/text/chat

Infers the next chat message for a given deployment. The deployment must reference either a prompt template with `input_mode` set to `chat`, a custom foundation model, or a curated foundation model. When a prompt template is referenced, the model used for the chat request is specified by the deployment's `base_model_id`. Chat parameters are derived from the prompt template's `model_parameters`. If a `serving_name` is provided, it must match the `serving_name` returned in the inference section at the time of deployment creation.

Related guides: - Deployment - Prompt template - Text chat

You can also use Deployment Chat Completions to achieve the same result.

Auditing

Calling this method generates the following auditing event.

• pm-20.foundation-model.send

Request

Request Body

DeploymentTextChatRequest

From a given prompt, infer the next chat message.

messages

Required *

array

The messages for this chat session.

If the deployment references a prompt template then `system` `role` can not be in `messages`. For such deployments, depending on the model, the `content` of `system` `role` may be from `system_prompt` of the prompt template, and will be automatically inserted into `messages`. As an example, depending on the model, if `system_prompt` of a prompt template is "You are Granite Chat, an AI language model developed by IBM. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.", a message with `system` `role` having `content` the same as `system_prompt` is inserted.

tools

array

Tool functions that can be called with the response.

tool_choice_option

string

Specify either `tool_choice_option` to allow the model to pick or `tool_choice` to force the model to call a tool.

Using `auto` means the model can pick between generating a message or calling one or more tools. Default is `auto`.

Using `none` means the model will not call any tool and instead generates a message.

Using `required` means the model must call one or more tools.

tool_choice

TextChatToolChoiceTool

Specifying a particular tool via `{"type": "function", "function": {"name": "my_function"}}` forces the model to call that tool. Specify either `tool_choice_option` to allow the model to pick or `tool_choice` to force the model to call a tool.

type

Required *

string

The tool type.

function

Required *

object

The named function.

context

string

If specified, `context` will be inserted into `messages`. Depending on the model, `context` may be inserted into the `content` with `system` `role`; or into the `content` of the last message of `user` `role`.

In the example, `context` "Today is Wednesday" is inserted as such `content` of `user` becomes "Today is Wednesday. Who are you and which day is tomorrow?"

chat_template_kwargs

object

Additional kwargs to pass to the chat template, described as a JSON Schema object. See the JSON Schema reference for documentation about the format.

frequency_penalty

number

Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

include_reasoning

boolean

Whether to include `reasoning_content` in the response. Default is `true`.

logit_bias

object

Increasing or decreasing probability of tokens being selected during generation; a positive bias makes a token more likely to appear, while a negative bias makes it less likely.

logprobs

boolean

Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.

top_logprobs

integer

An integer specifying the number of most likely tokens to return at each token position, each with an associated log probability. The option `logprobs` must be set to `true` if this parameter is used.

max_completion_tokens

integer

The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. Set to 0 for the model's configured max generated tokens.

max_tokens

integer

The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. Set to 0 for the model's configured max generated tokens.

This value is now deprecated in favor of `max_completion_tokens`. If specified together with `max_completion_tokens`, `max_tokens` will be ignored.

n

integer

How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Keep n as 1 to minimize costs.

presence_penalty

number

Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

reasoning_effort

string

A lower reasoning effort can result in faster responses, fewer tokens used, and shorter `reasoning_content` in the responses. Supported values are `low`, `medium`, and `high`.

response_format

TextChatResponseFormat

The chat response format parameters.

type

Required *

string

Used to enable JSON mode, which guarantees the message the model generates is valid JSON.

Important: when using JSON mode, you must also instruct the model to produce JSON yourself via a system or user message. Without this, the model may generate an unending stream of whitespace until the generation reaches the token limit, resulting in a long-running and seemingly "stuck" request. Also note that the message content may be partially cut off if `finish_reason="length"`, which indicates the generation exceeded `max_tokens` or the conversation exceeded the max context length.

• `text` : Will not create a JSON structured output.
• `json_object` : User will get a JSON structured output, but the schema is not guranteed.
• `json_schema` : User needs to declare the `json_schema` object to provide the desired JSON schema in order to get a matched response.

json_schema

object

User-defined JSON schema object with optional parameters to enforce structured output.

Important: this field will only applied when `response_format`.`type` is set to `json_schema`.

seed

integer

Random number generator seed to use in sampling mode for experimental repeatability.

stop

array

Stop sequences are one or more strings which will cause the text generation to stop if/when they are produced as part of the output. Stop sequences encountered prior to the minimum number of tokens being generated will be ignored.

stream

boolean

If set to true, this operation will return the output tokens as a stream of events, and usage is always included in the response.

temperature

number

What sampling temperature to use,. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.

We generally recommend altering this or `top_p` but not both.

top_p

number

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

We generally recommend altering this or `temperature` but not both.

time_limit

integer

Time limit in milliseconds - if not completed within this time, generation will stop. The text generated so far will be returned along with the `TIME_LIMIT`` stop reason. Depending on the users plan, and on the model being used, there may be an enforced maximum time limit.

Response

200 - Successful operation

id

string

A unique identifier for the chat completion.

model_id

string

The model used for the chat completion.

model

string

This field is a duplicate of `model_id` and is provided in order to provide better compatibility with other APIs.

model_version

string

The model version (using semantic versioning) if set.

created

integer

The Unix timestamp (in seconds) of when the chat completion was created.

created_at

string

The time when the response was created in ISO 8601 format.

usage

TextChatUsage

Usage statistics for the completion request.

moderations

ModerationResults

The result of any detected moderations.

choices

array

A list of chat completion choices. Can be more than one if `n` is greater than 1.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Infer text chat event stream

POST /ml/v1/deployments/{id_or_name}/text/chat_stream

Infers the next chat message for a given deployment. This operation will return the output tokens as a stream of events. The deployment must reference either a prompt template with `input_mode` set to `chat`, a custom foundation model, or a curated foundation model. When a prompt template is referenced, the model used for the chat request is specified by the deployment's `base_model_id`. Chat parameters are derived from the prompt template's `model_parameters`. If a `serving_name` is provided, it must match the `serving_name` returned in the inference section at the time of deployment creation. Related guides: - Deployment - Prompt template - Text chat

You can also use Deployment Chat Completions with `stream` option set to true to achieve the same result.

Auditing

Calling this method generates the following auditing event.

• pm-20.foundation-model.send

Request

Request Body

DeploymentTextChatRequest

From a given prompt, infer the next chat message in a server-sent events (SSE) stream.

messages

Required *

array

The messages for this chat session.

If the deployment references a prompt template then `system` `role` can not be in `messages`. For such deployments, depending on the model, the `content` of `system` `role` may be from `system_prompt` of the prompt template, and will be automatically inserted into `messages`. As an example, depending on the model, if `system_prompt` of a prompt template is "You are Granite Chat, an AI language model developed by IBM. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.", a message with `system` `role` having `content` the same as `system_prompt` is inserted.

tools

array

Tool functions that can be called with the response.

tool_choice_option

string

Specify either `tool_choice_option` to allow the model to pick or `tool_choice` to force the model to call a tool.

Using `auto` means the model can pick between generating a message or calling one or more tools. Default is `auto`.

Using `none` means the model will not call any tool and instead generates a message.

Using `required` means the model must call one or more tools.

tool_choice

TextChatToolChoiceTool

Specifying a particular tool via `{"type": "function", "function": {"name": "my_function"}}` forces the model to call that tool. Specify either `tool_choice_option` to allow the model to pick or `tool_choice` to force the model to call a tool.

type

Required *

string

The tool type.

function

Required *

object

The named function.

context

string

If specified, `context` will be inserted into `messages`. Depending on the model, `context` may be inserted into the `content` with `system` `role`; or into the `content` of the last message of `user` `role`.

In the example, `context` "Today is Wednesday" is inserted as such `content` of `user` becomes "Today is Wednesday. Who are you and which day is tomorrow?"

chat_template_kwargs

object

Additional kwargs to pass to the chat template, described as a JSON Schema object. See the JSON Schema reference for documentation about the format.

frequency_penalty

number

Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

include_reasoning

boolean

Whether to include `reasoning_content` in the response. Default is `true`.

logit_bias

object

Increasing or decreasing probability of tokens being selected during generation; a positive bias makes a token more likely to appear, while a negative bias makes it less likely.

logprobs

boolean

Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.

top_logprobs

integer

An integer specifying the number of most likely tokens to return at each token position, each with an associated log probability. The option `logprobs` must be set to `true` if this parameter is used.

max_completion_tokens

integer

The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. Set to 0 for the model's configured max generated tokens.

max_tokens

integer

The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. Set to 0 for the model's configured max generated tokens.

This value is now deprecated in favor of `max_completion_tokens`. If specified together with `max_completion_tokens`, `max_tokens` will be ignored.

n

integer

How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Keep n as 1 to minimize costs.

presence_penalty

number

Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

reasoning_effort

string

A lower reasoning effort can result in faster responses, fewer tokens used, and shorter `reasoning_content` in the responses. Supported values are `low`, `medium`, and `high`.

response_format

TextChatResponseFormat

The chat response format parameters.

type

Required *

string

Used to enable JSON mode, which guarantees the message the model generates is valid JSON.

Important: when using JSON mode, you must also instruct the model to produce JSON yourself via a system or user message. Without this, the model may generate an unending stream of whitespace until the generation reaches the token limit, resulting in a long-running and seemingly "stuck" request. Also note that the message content may be partially cut off if `finish_reason="length"`, which indicates the generation exceeded `max_tokens` or the conversation exceeded the max context length.

• `text` : Will not create a JSON structured output.
• `json_object` : User will get a JSON structured output, but the schema is not guranteed.
• `json_schema` : User needs to declare the `json_schema` object to provide the desired JSON schema in order to get a matched response.

json_schema

object

User-defined JSON schema object with optional parameters to enforce structured output.

Important: this field will only applied when `response_format`.`type` is set to `json_schema`.

seed

integer

Random number generator seed to use in sampling mode for experimental repeatability.

stop

array

Stop sequences are one or more strings which will cause the text generation to stop if/when they are produced as part of the output. Stop sequences encountered prior to the minimum number of tokens being generated will be ignored.

stream

boolean

If set to true, this operation will return the output tokens as a stream of events, and usage is always included in the response.

temperature

number

What sampling temperature to use,. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.

We generally recommend altering this or `top_p` but not both.

top_p

number

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

We generally recommend altering this or `temperature` but not both.

time_limit

integer

Time limit in milliseconds - if not completed within this time, generation will stop. The text generated so far will be returned along with the `TIME_LIMIT`` stop reason. Depending on the users plan, and on the model being used, there may be an enforced maximum time limit.

Response

200 - Successful operation (`Content-Type: text/event-stream`).

400 -

401 -

403 -

404 -

Time series forecast

POST /ml/v1/deployments/{id_or_name}/time_series/forecast

Generate forecasts, or predictions for future time points, given historical time series data.

Auditing

Calling this method generates the following auditing event.

• pm-20.time-series-forecast.send

Request

Request Body

DeploymentTSForecastResource

The forecast request.

data

Required *

object

A payload of data matching `schema`. We assume the following about your data: * All timeseries are of equal length and are uniform in nature (the time difference between two successive rows is constant). This implies that there are no missing rows of data; * The data meet the minimum model-dependent historical context length which can be any number of rows per timeseries;

Note that the example payloads shown are for illustration purposes only. An actual payload would necessary be much larger to meet minimum model-specific context lengths.

parameters

DeploymentTSForecastParameters

The parameters for the forecast request.

prediction_length

integer

The prediction length for the forecast. The service will return this many periods beyond the last timestamp in the inference data payload. If specified, `prediction_length` must be an integer >=1 and no more than the model default prediction length. When omitted the model default prediction_length will be used.

inference_batch_size

integer

The batch size used during inference. When multiple time series are present, the inference will be conducted in batches. If not specified, the model default batch size will be used.

schema

Required *

TSForecastInputSchema

Contains metadata about your timeseries data input.

timestamp_column

Required *

string

A valid column in the data that should be treated as the timestamp. Although not absolutely necessary, if using calendar dates (simple integer time offsets are also allowed), users should consider using a format such as ISO 8601 that includes a UTC offset (e.g., '2024-10-18T01:09:21.454746+00:00'). This will avoid potential issues such as duplicate dates appearing due to daylight savings change overs. There are many date formats in existence and inferring the correct one can be a challenge so please do consider adhering to ISO 8601.

id_columns

array

Columns that define a unique key for timeseries. This is similar to a compound primary key in a database table.

freq

string

A frequency indicator for the given timestamp_column. See https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#period-aliases for a description of the allowed values. If not provided, we will attempt to infer it from the data.

target_columns

array

An array of column headings which constitute the target variables in the data. These are the data that will be forecasted.

future_data

object

Exogenous or supporting features that extend into the forecasting horizon (e.g., a weather forecast or calendar of special promotions) which are known in advance. `future_data` would be in the same format as `data` except that all timestamps would be in the forecast horizon and it would not include previously specified `target_columns`.

Response

200 - Successful operation

model_id

string

The model used to generate the forecast.

created_at

string

The time when the response was created in ISO 8601 format.

results

array

The list of prediction results. There will be a forecast for each time series in the input data. The `prediction_length` field in the request specifies the number of predictions in the results. The actual number of rows in the results will be equal to the `prediction length` multiplied by the number of unique ids in `id_columns`. The `timestamp_column` field in the request indicates the name of the timestamp column in the results.

input_data_points

integer

The number of input data points (number of rows in `data` * number of input columns in `data`).

output_data_points

integer

The number of forecasted data points (`prediction_length` * number of `target_columns` * number of unique ids in `id_columns`).

400 -

401 -

403 -

404 -

Execute AI service

POST /ml/v4/deployments/{id_or_name}/ai_service

Execute an inference call for the AI service deployment. If a `serving_name` is used then it must match the `serving_name` that is returned in the `inference` section when the deployment was created.

Auditing

Calling this method generates the following auditing event.

• pm-20.ai_service.send

Request

Request Body

DeploymentAIServiceRequest

Call the deployed AI service with the json payload.

Response

200 - Successful operation

400 -

401 -

403 -

404 -

Execute AI service stream

POST /ml/v4/deployments/{id_or_name}/ai_service_stream

Execute an inference call for the AI service deployment. If a `serving_name` is used then it must match the `serving_name` that is returned in the `inference` section when the deployment was created.

Auditing

Calling this method generates the following auditing event.

• pm-20.ai_service.send

Request

Request Body

DeploymentAIServiceRequest

Call the deployed AI service with the json payload.

Response

200 - Successful operation

400 -

401 -

403 -

404 -

POST /ml/v4/trainings

Create a new watsonx.ai training in a project or a space.

In order to deploy the tuned model you need to follow the following steps:

1. Create a WML model asset, in a space or a project,
 by providing the `request.json` as shown below:
   ```
   curl -X POST "https://{cpd_cluster}/ml/v4/models?version=2024-01-29" \
     -H "Authorization: Bearer <replace with your token>" \
     -H "content-type: application/json" \
     --data '{
        "name": "replace_with_a_meaningful_name",
        "space_id": "replace_with_your_space_id",
        "type": "prompt_tune_1.0",
        "software_spec": {
          "name": "watsonx-textgen-fm-1.0"
        },
        "metrics": [ from the training job ],
        "training": {
          "id": "05859469-b25b-420e-aefe-4a5cb6b595eb",
          "base_model": {
            "model_id": "google/flan-t5-xl"
          },
          "task_id": "generation",
          "verbalizer": "Input:  Output:"
        },
        "training_data_references": [
          {
            "connection": {
              "id": "20933468-7e8a-4706-bc90-f0a09332b263"
            },
            "id": "file_to_tune1.json",
            "location": {
              "bucket": "wxproject-donotdelete-pr-xeyivy0rx3vrbl",
              "path": "file_to_tune1.json"
            },
            "type": "connection_asset"
          }
        ]
      }'
   ```


   **Notes:**

   1. If you used the training request field `auto_update_model: true`
   then you can skip this step as the model will have been saved at
   the end of the training job.
   1. Rather than creating the payload for the model you can use the
      generated `request.json` that was stored in the `results_reference`
      field, look for the path in the field
      `entity.results_reference.location.model_request_path`.
   1. The model `type` must be `prompt_tune_1.0`.
   1. The software spec name must be `watsonx-textgen-fm-1.0`.

1. Create a tuned model deployment as described in the
 [create deployment documentation](#create-deployment).

Auditing

Calling this method generates the following auditing event.

• pm-20.training.create

Request

Request Body

TrainingResourcePrototype

name

Required *

string

The name of the training.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

description

string

A description of the training.

tags

Tags

A list of tags for this resource.

training_data_references

array

Training datasets.

custom

Custom

User defined properties specified as key-value pairs.

auto_update_model

boolean

If set to `true` then the result of the training, if successful, will be uploaded to the repository as a model.

results_reference

Required *

object

The training results. Normally this is specified as `type=container` (Service) or `type=fs` (Software) which means that it is stored in the space or project.

id

string

Item identification inside a collection.

type

Required *

string

The data source type like `connection_asset`, `container` (Service) or `fs` (Software).

connection

object

Contains a set of fields specific to each connection. See here for details about specifying connections.

location

Required *

object

Contains a set of fields that describe the location of the data with respect to the `connection`.

Response

201 - The training job has been created.

metadata

ResourceMeta

Common metadata for a simple resource.

entity

TrainingResourceEntity

Status of the training job.

400 -

401 -

403 -

404 -

Retrieve the list of trainings

GET /ml/v4/trainings

Retrieve the list of trainings for the specified space or project.

Auditing

Calling this method generates the following auditing event.

• pm-20.training.list

Request

Path Parameters

Query Parameters

start

string

Token required for token-based pagination. This token cannot be determined by end user. It is generated by the service and it is set in the href available in the `next` field.

limit

integer

How many resources should be returned. By default limit is 100. Max limit allowed is 200.

total_count

boolean

Compute the total count. May have performance impact.

tag.value

string

Return only the resources with the given tag value.

state

string

Filter based on on the training job state.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

Response

200 - OK.

total_count

integer

The total number of resources. Computed explicitly only when `total_count=true` query parameter is present. This is in order to avoid performance penalties.

limit

integer

The number of items to return in each page.

first

PaginationFirst

The reference to the first item in the current page.

next

PaginationNext

A reference to the first item of the next page, if any.

resources

array

The training resources.

system

object

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Retrieve the training

GET /ml/v4/trainings/{training_id}

Retrieve the training with the specified identifier.

Auditing

Calling this method generates the following auditing event.

• pm-20.training.get

Request

Response

200 - OK.

metadata

ResourceMeta

Common metadata for a simple resource.

entity

TrainingResourceEntity

Status of the training job.

400 -

401 -

403 -

404 -

Cancel or delete the training

DELETE /ml/v4/trainings/{training_id}

Cancel or delete the specified training, once deleted all trace of the job is gone.

Auditing

Calling this method generates the following auditing event.

• pm-20.training.delete

Request

Path Parameters

Query Parameters

hard_delete

boolean

Set to true in order to also delete the job or request metadata.

Response

204 - Training cancelled.

400 -

401 -

403 -

404 -

Generate embeddings

POST /ml/v1/text/embeddings

Generate embeddings from text input.

See the documentation for a description of text embeddings.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-embeddings.send

Request

Request Body

EmbeddingsRequest

The text input for a given model to be used to generate the embeddings.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

model_id

Required *

string

The `id` of the model to be used for this request. Please refer to the list of models.

inputs

Required *

array

The input text.

parameters

EmbeddingParameters

Parameters for text embedding requests.

truncate_input_tokens

integer

Represents the maximum number of tokens accepted per input.

This can be used to avoid requests failing due to input being longer than configured limits. If the text is truncated, then it truncates the end of the input (on the right), so the start of the input will remain the same.

If this value exceeds the `maximum sequence length` (refer to the documentation to find this value for the model) then the call will fail if the total number of tokens exceeds the `maximum sequence length`.

return_options

object

The return options for text embeddings.

crypto

CryptoInfo

To enable encryption, configure credentials for your chosen keys management service. For details, see Encrypting inference requests.

key_ref

Required *

string

Identifier of the DEK in the chosen keys management service.

• IBM Key Protect - full CRN (e.g. `crn:v1:bluemix:public:kms:us-south:a/12345:<instance_id>:key:<root_key_id>:wdek:<ciphertext>`) This field is required, but its exact format depends on the selected `keys_manager`. No strict pattern is enforced here - validation will be performed by the keys management service.

Response

200 - Successful operation

model_id

string

The `id` of the model to be used for this request. Please refer to the list of models.

results

array

The embedding values for a given text.

created_at

string

The time when the response was created in ISO 8601 format.

input_token_count

integer

The number of input tokens that were consumed.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Detect text similarities

POST /ml/v1/text/similarities

Detect similarities between source strings and target strings.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-similarities.send

Request

Request Body

SimilarityRequest

The strings to be used to detect text similarities.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

model_id

Required *

string

The `id` of the model to be used for this request. Please refer to the list of models.

source_text

Required *

string

A source text.

target_texts

Required *

array

The target texts.

parameters

SimilarityParameters

Parameters for text similarity.

truncate_input_tokens

integer

Represents the maximum number of tokens accepted per input.

This can be used to avoid requests failing due to input being longer than configured limits. If the text is truncated, then it truncates the end of the input (on the right), so the start of the input will remain the same.

If this value exceeds the `maximum sequence length` (refer to the documentation to find this value for the model) then the call will fail if the total number of tokens exceeds the `maximum sequence length`.

return_options

object

The return options for text similarity.

Response

200 - Successful operation

model_id

string

The `id` of the model to be used for this request. Please refer to the list of models.

results

array

The similarity scores.

created_at

string

The time when the response was created in ISO 8601 format.

input_token_count

integer

The number of input tokens that were consumed.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Generate rerank

POST /ml/v1/text/rerank

Rerank texts based on some queries.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-rerank.send

Request

Request Body

RerankRequest

The input texts and the queries for reranking.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

model_id

Required *

string

The `id` of the model to be used for this request. Please refer to the list of models.

inputs

Required *

array

The rank input strings.

query

Required *

string

The rank query.

parameters

RerankParameters

The properties used for reranking.

truncate_input_tokens

integer

Represents the maximum number of tokens accepted per input.

This can be used to avoid requests failing due to input being longer than configured limits. If the text is truncated, then it truncates the end of the input (on the right), so the start of the input will remain the same.

If this value exceeds the `maximum sequence length` (refer to the documentation to find this value for the model) then the call will fail if the total number of tokens exceeds the `maximum sequence length`.

return_options

object

The return options for text reranking.

crypto

CryptoInfo

To enable encryption, configure credentials for your chosen keys management service. For details, see Encrypting inference requests.

key_ref

Required *

string

Identifier of the DEK in the chosen keys management service.

• IBM Key Protect - full CRN (e.g. `crn:v1:bluemix:public:kms:us-south:a/12345:<instance_id>:key:<root_key_id>:wdek:<ciphertext>`) This field is required, but its exact format depends on the selected `keys_manager`. No strict pattern is enforced here - validation will be performed by the keys management service.

Response

200 - Successful operation

model_id

string

The `id` of the model to be used for this request. Please refer to the list of models.

model_version

string

The model version (using semantic versioning) if set.

results

array

The ranked results.

created_at

string

The time when the response was created in ISO 8601 format.

input_token_count

integer

The number of input tokens that were consumed.

query

string

The rank query, if requested.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Create a geospatial transformation

POST /ml/v1/geospatial/transformations

Create a geospatial transformation from inputs using a given model.

Request

Request Body

GeospatialTransformationRequest

The details of the inputs to transform with the given model.

model_id

Required *

string

The `id` of the model to be used for this request.

inputs

Required *

array

The input text.

output

Required *

GeospatialTransformationData

A data location of the input or output data.

url

string

A signed url for the data in COS or elsewhere. This url will be used to download if this is input, or upload if this is output.

space_id

Required *

string

The space that contains the resource.

Response

201 - Created. The `Content-Location` header will contain the URI reference to the created resource.

metadata

GeospatialTransformationMetadata

Common metadata for a simple resource.

entity

GeospatialTransformationResponseEntity

The inputs and outputs required to create a transformation for the given model.

400 -

401 -

403 -

404 -

GET /ml/v1/geospatial/transformations

Retrieve the list of geospatial transformation requests for the specified space or project.

This operation does not save the history, any requests that were deleted or purged will not appear in this list.

Request

Path Parameters

Query Parameters

version

Required *

string

The version date for the API of the form `YYYY-MM-DD`.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

start

string

Token required for token-based pagination. This token cannot be determined by end user. It is generated by the service and it is set in the href available in the `next` field.

limit

integer

How many resources should be returned. By default limit is 100. Max limit allowed is 200.

Response

200 - OK.

total_count

integer

The total number of resources. Computed explicitly only when `total_count=true` query parameter is present. This is in order to avoid performance penalties.

limit

integer

The number of items to return in each page.

first

PaginationFirst

The reference to the first item in the current page.

next

PaginationNext

A reference to the first item of the next page, if any.

resources

array

A list of resources.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Get a geospatial transformation

GET /ml/v1/geospatial/transformations/{id}

Get a geospatial transformation.

Request

Response

200 - OK.

metadata

GeospatialTransformationMetadata

Common metadata for a simple resource.

entity

GeospatialTransformationResponseEntity

The inputs and outputs required to create a transformation for the given model.

400 -

401 -

403 -

404 -

Delete a geospatial transformation

DELETE /ml/v1/geospatial/transformations/{id}

Delete a geospatial transformation.

Request

Response

204 - Deleted.

400 -

401 -

403 -

404 -

Create a new AutoAI RAG run

POST /ml/v1/autoai/rags

Create a new AutoAI RAG that will find the best RAG pattern from the data that is provided in the request.

Request

Request Body

AutoAIRAGRequest

The details of the AutoAI RAG run with the data used to find the best RAG patterns.

name

Required *

string

The name of the job.

description

string

The description of the job.

tags

Tags

A list of tags for this resource.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

hardware_spec

Required *

HardwareSpec

A hardware specification.

id

string

The id of the hardware specification.

rev

string

The revision of the hardware specification.

name

string

The name of the hardware specification.

num_nodes

integer

The number of nodes applied to a computation.

parameters

AutoAIRAGParameters

The parameters for an AutoAI RAG run.

constraints

object

The constraint parameters for an AutoAI RAG run.

optimization

object

The optimization parameters for an AutoAI RAG run.

deployment

object

Best pattern deployment related properties.

output_logs

boolean

Request that the output logs are also stored as part of the result.

input_data_references

array

A set of input data references.

test_data_references

array

A set of test data references.

knowledge_base_references

array

A set of existing knowledge base references. Applicable to multi-retrieval scenarios. Either `input_data_references` or `knowledge_base_references` must be provided.

vector_store_references

array

A set of vector store references.

custom

Custom

User defined properties specified as key-value pairs.

results_reference

Required *

object

The training results. Normally this is specified as `type=container` (Service) or `type=fs` (Software) which means that it is stored in the space or project.

id

string

Item identification inside a collection.

type

Required *

string

The data source type like `connection_asset`, `container` (Service) or `fs` (Software).

connection

object

Contains a set of fields specific to each connection. See here for details about specifying connections.

location

Required *

object

Contains a set of fields that describe the location of the data with respect to the `connection`.

Response

201 - Created.

metadata

AutoAIRAGMetadata

Common metadata for a simple resource.

entity

AutoAIRAGResponseEntity

The request entity for an AutoAI RAG run.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

GET /ml/v1/autoai/rags

Retrieve the list of AutoAI RAG requests for the specified space or project.

This operation does not save the history, any requests that were deleted or purged will not appear in this list.

Request

Path Parameters

Query Parameters

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

start

string

Token required for token-based pagination. This token cannot be determined by end user. It is generated by the service and it is set in the href available in the `next` field.

limit

integer

How many resources should be returned. By default limit is 100. Max limit allowed is 200.

Response

200 - OK.

total_count

integer

The total number of resources. Computed explicitly only when `total_count=true` query parameter is present. This is in order to avoid performance penalties.

limit

integer

The number of items to return in each page.

first

PaginationFirst

The reference to the first item in the current page.

next

PaginationNext

A reference to the first item of the next page, if any.

resources

array

A list of training definitions.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Get an AutoAI RAG run

GET /ml/v1/autoai/rags/{id}

Get the results of an AutoAI RAG run, or details if the job failed.

Request

Response

200 - OK.

metadata

AutoAIRAGMetadata

Common metadata for a simple resource.

entity

AutoAIRAGResponseEntity

The request entity for an AutoAI RAG run.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Cancel or delete an AutoAI RAG run

DELETE /ml/v1/autoai/rags/{id}

Cancel or delete the specified AutoAI RAG run, once deleted all trace of the run job is gone.

Request

Path Parameters

Query Parameters

hard_delete

boolean

Set to true in order to also delete the job or request metadata.

Response

204 - Deleted.

400 -

401 -

403 -

404 -

Start a text extraction request

POST /ml/v1/text/extractions

Start a request to extract text and metadata from documents.

See the documentation for a description of text extraction.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-extraction.send

Request

Request Body

TextExtractionRequest

The input for the text extraction request.

document_reference

Required *

TextExtractionDataReference

A reference to data.

type

Required *

string

The data source type.

connection

object

Contains a set of location fields specific to each data source.

location

object

Contains a set of fields specific to each connection.

results_reference

Required *

TextExtractionDataReference

A reference to data.

type

Required *

string

The data source type.

connection

object

Contains a set of location fields specific to each data source.

location

object

Contains a set of fields specific to each connection.

parameters

TextExtractionParameters

The parameters for the text extraction.

requested_outputs

array

A list of requested outputs.

mode

string

Which processing mode to use for this request.

ocr_mode

string

If OCR should be used when processing a document. An empty value allows the service to select the best option for your processing mode.

• `enabled`: OCR is run on embedded images, OCR is only run if no programmatic text could be extracted from the area.
• `disabled`: OCR is not run, no information is extracted from images or scanned documents.
• `forced`: Document processing will take a picture of the page and run OCR across it, this applies to all documents even purely programmatic ones.

languages

array

Set of languages to be expected in the document. The language codes follow `ISO 639` where possible. See the documentation for the currently supported languages.

auto_rotation_correction

boolean

Should the service attempt to fix a rotated page or image.

create_embedded_images

string

If the service should return embedded images in the output markdown and JSON formats.

output_dpi

number

The DPI for extracted images.

output_tokens

boolean

Disables returning the individual tokens.

kvp_mode

string

Which version of KVP should be used when processing, if not set then KVP is disabled. See the documentation for more information about the available modes.

semantic_config

object

Properties related to semantic config.

custom

Custom

User defined properties specified as key-value pairs.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

Response

201 - Created. The `Content-Location` header will contain the URI reference to the created resource.

metadata

TextExtractionMetadata

Common metadata for a simple resource.

entity

object

The parameters for the text extraction.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

GET /ml/v1/text/extractions

Retrieve the list of text extraction requests for the specified space or project.

This operation does not save the history, any requests that were deleted or purged will not appear in this list.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-extraction.list

Request

Path Parameters

Query Parameters

version

Required *

string

The version date for the API of the form `YYYY-MM-DD`.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

start

string

Token required for token-based pagination. This token cannot be determined by end user. It is generated by the service and it is set in the href available in the `next` field.

limit

integer

How many resources should be returned. By default limit is 100. Max limit allowed is 200.

Response

200 - OK.

total_count

integer

The total number of resources. Computed explicitly only when `total_count=true` query parameter is present. This is in order to avoid performance penalties.

limit

integer

The number of items to return in each page.

first

PaginationFirst

The reference to the first item in the current page.

next

PaginationNext

A reference to the first item of the next page, if any.

resources

array

A list of resources.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

GET /ml/v1/text/extractions/{id}

Retrieve the text extraction request with the specified identifier.

Note that there is a retention period of 2 days. If this retention period is exceeded then the request will be deleted and the results no longer available. In this case this operation will return `404`.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-extraction.get

Request

Response

200 - OK.

metadata

TextExtractionMetadata

Common metadata for a simple resource.

entity

object

The parameters for the text extraction.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Delete the request

DELETE /ml/v1/text/extractions/{id}

Cancel the specified text extraction request and delete any associated results.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-extraction.delete

Request

Path Parameters

Query Parameters

hard_delete

boolean

Set to true in order to also delete the job or request metadata.

Response

204 - Request deleted.

400 -

401 -

403 -

404 -

Create an image

POST /ml/v1/text/image

Create an image from a textual description.

See the documentation for a description of image generation.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-image.send

Request

Request Body

TextImageRequest

The input for the image generation request.

Set the HTTP header `Accept: image/png` in order to get the image in `png` format.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

model_id

Required *

string

The `id` of the model to be used for this request.

Please refer to the list of models.

input

Required *

string

The textual description of the image to be generated.

parameters

TextImageParameters

Properties that control the image generation.

height

integer

The height of the generated image.

Note that only certain heights may be supported depending on the model being used.

width

integer

The width of the generated image.

Note that only certain widths may be supported depending on the model being used.

moderations

ModerationsInput

Properties that control the moderations, for usages such as `Hate and profanity` (HAP) and `Personal identifiable information` (PII) filtering. This list can be extended with new types of moderations.

hap

object

The properties for the moderation. Each type of moderation may have additional properties that are specific to that moderation.

pii

object

The properties for the moderation. Each type of moderation may have additional properties that are specific to that moderation.

Response

200 - Ok.

400 -

401 -

403 -

404 -

POST /ml/v1/chat/completions

Infer the next tokens for a given deployed model with a set of PARAMS1.

If `stream` is true, this operation will return the output tokens in a server-sent events (SSE) stream.

Auditing

Calling this method generates the following auditing event.

• chat-completions.send

Request

Request Body

ChatCompletionsRequest

From a given prompt, infer the next tokens.

model

Required *

string

The model to use for the chat completion.

Please refer to the list of models.

messages

Required *

array

The messages for this chat session.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

tools

array

Tool functions that can be called with the response.

tool_choice_option

string

Specify either `tool_choice_option` to allow the model to pick or `tool_choice` to force the model to call a tool.

Using `auto` means the model can pick between generating a message or calling one or more tools. Default is `auto`.

Using `none` means the model will not call any tool and instead generates a message.

Using `required` means the model must call one or more tools.

tool_choice

TextChatToolChoiceTool

Specifying a particular tool via `{"type": "function", "function": {"name": "my_function"}}` forces the model to call that tool. Specify either `tool_choice_option` to allow the model to pick or `tool_choice` to force the model to call a tool.

type

Required *

string

The tool type.

function

Required *

object

The named function.

chat_template_kwargs

object

Additional kwargs to pass to the chat template, described as a JSON Schema object. See the JSON Schema reference for documentation about the format.

frequency_penalty

number

Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

include_reasoning

boolean

Whether to include `reasoning_content` in the response. Default is `true`.

logit_bias

object

Increasing or decreasing probability of tokens being selected during generation; a positive bias makes a token more likely to appear, while a negative bias makes it less likely.

logprobs

boolean

Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.

top_logprobs

integer

An integer specifying the number of most likely tokens to return at each token position, each with an associated log probability. The option `logprobs` must be set to `true` if this parameter is used.

max_completion_tokens

integer

The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. Set to 0 for the model's configured max generated tokens.

max_tokens

integer

The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. Set to 0 for the model's configured max generated tokens.

This value is now deprecated in favor of `max_completion_tokens`. If specified together with `max_completion_tokens`, `max_tokens` will be ignored.

n

integer

How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Keep n as 1 to minimize costs.

presence_penalty

number

Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

reasoning_effort

string

A lower reasoning effort can result in faster responses, fewer tokens used, and shorter `reasoning_content` in the responses. Supported values are `low`, `medium`, and `high`.

response_format

TextChatResponseFormat

The chat response format parameters.

type

Required *

string

Used to enable JSON mode, which guarantees the message the model generates is valid JSON.

Important: when using JSON mode, you must also instruct the model to produce JSON yourself via a system or user message. Without this, the model may generate an unending stream of whitespace until the generation reaches the token limit, resulting in a long-running and seemingly "stuck" request. Also note that the message content may be partially cut off if `finish_reason="length"`, which indicates the generation exceeded `max_tokens` or the conversation exceeded the max context length.

• `text` : Will not create a JSON structured output.
• `json_object` : User will get a JSON structured output, but the schema is not guranteed.
• `json_schema` : User needs to declare the `json_schema` object to provide the desired JSON schema in order to get a matched response.

json_schema

object

User-defined JSON schema object with optional parameters to enforce structured output.

Important: this field will only applied when `response_format`.`type` is set to `json_schema`.

seed

integer

Random number generator seed to use in sampling mode for experimental repeatability.

stop

array

Stop sequences are one or more strings which will cause the text generation to stop if/when they are produced as part of the output. Stop sequences encountered prior to the minimum number of tokens being generated will be ignored.

stream

boolean

If set to true, this operation will return the output tokens as a stream of events, and usage is always included in the response.

temperature

number

What sampling temperature to use,. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.

We generally recommend altering this or `top_p` but not both.

top_p

number

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

We generally recommend altering this or `temperature` but not both.

time_limit

integer

Time limit in milliseconds - if not completed within this time, generation will stop. The text generated so far will be returned along with the `TIME_LIMIT`` stop reason. Depending on the users plan, and on the model being used, there may be an enforced maximum time limit.

Response

200 - Successful operation. `Content-Type: text/event-stream` if `stream` is true.

id

string

A unique identifier for the chat completion.

model_id

string

The model used for the chat completion.

model

string

This field is a duplicate of `model_id` and is provided in order to provide better compatibility with other APIs.

model_version

string

The model version (using semantic versioning) if set.

created

integer

The Unix timestamp (in seconds) of when the chat completion was created.

created_at

string

The time when the response was created in ISO 8601 format.

usage

TextChatUsage

Usage statistics for the completion request.

moderations

ModerationResults

The result of any detected moderations.

choices

array

A list of chat completion choices. Can be more than one if `n` is greater than 1.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Infer text

POST /ml/v1/text/chat

Infer the next tokens for a given deployed model with a set of PARAMS2.

You can also use /v1/chat/completions to achieve the same result.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-chat.send

Request

Request Body

TextChatRequest

From a given prompt, infer the next tokens.

model_id

Required *

string

The model to use for the chat completion.

Please refer to the list of models.

model

string

This is an alias for `model_id` and can be used instead of `model_id`.

The field `model_id` is marked as mandatory but it can be replaced by the `model` field for compatibility with other APIs.

messages

Required *

array

The messages for this chat session.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

moderations

Moderations

Properties that control the moderations, for usages such as `Hate and profanity` (HAP) and `Personal identifiable information` (PII) filtering. This list can be extended with new types of moderations.

hap

object

The properties for the moderation. Each type of moderation may have additional properties that are specific to that moderation.

pii

object

The properties for the moderation. Each type of moderation may have additional properties that are specific to that moderation.

granite_guardian

object

The properties specific to Granite Guardian. This detector is in beta and may change.

input_ranges

array

If set, then only these ranges will be applied to the moderations. This is useful in the case that certain parts of the input text have already been checked.

tools

array

Tool functions that can be called with the response.

tool_choice_option

string

Specify either `tool_choice_option` to allow the model to pick or `tool_choice` to force the model to call a tool.

Using `auto` means the model can pick between generating a message or calling one or more tools. Default is `auto`.

Using `none` means the model will not call any tool and instead generates a message.

Using `required` means the model must call one or more tools.

tool_choice

TextChatToolChoiceTool

Specifying a particular tool via `{"type": "function", "function": {"name": "my_function"}}` forces the model to call that tool. Specify either `tool_choice_option` to allow the model to pick or `tool_choice` to force the model to call a tool.

type

Required *

string

The tool type.

function

Required *

object

The named function.

guided_choice

array

If specified, the output will be exactly one of the choices.

guided_regex

string

If specified, the output will follow the regex pattern.

guided_grammar

string

If specified, the output will follow the context free grammar.

guided_json

object

If specified, the output will follow the JSON schema. See the JSON Schema reference for documentation about the format.

crypto

CryptoInfo

To enable encryption, configure credentials for your chosen keys management service. For details, see Encrypting inference requests.

key_ref

Required *

string

Identifier of the DEK in the chosen keys management service.

• IBM Key Protect - full CRN (e.g. `crn:v1:bluemix:public:kms:us-south:a/12345:<instance_id>:key:<root_key_id>:wdek:<ciphertext>`) This field is required, but its exact format depends on the selected `keys_manager`. No strict pattern is enforced here - validation will be performed by the keys management service.

chat_template_kwargs

object

Additional kwargs to pass to the chat template, described as a JSON Schema object. See the JSON Schema reference for documentation about the format.

frequency_penalty

number

Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

include_reasoning

boolean

Whether to include `reasoning_content` in the response. Default is `true`.

logit_bias

object

Increasing or decreasing probability of tokens being selected during generation; a positive bias makes a token more likely to appear, while a negative bias makes it less likely.

logprobs

boolean

Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.

top_logprobs

integer

An integer specifying the number of most likely tokens to return at each token position, each with an associated log probability. The option `logprobs` must be set to `true` if this parameter is used.

max_completion_tokens

integer

The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. Set to 0 for the model's configured max generated tokens.

max_tokens

integer

The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. Set to 0 for the model's configured max generated tokens.

This value is now deprecated in favor of `max_completion_tokens`. If specified together with `max_completion_tokens`, `max_tokens` will be ignored.

n

integer

How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Keep n as 1 to minimize costs.

presence_penalty

number

Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

reasoning_effort

string

A lower reasoning effort can result in faster responses, fewer tokens used, and shorter `reasoning_content` in the responses. Supported values are `low`, `medium`, and `high`.

response_format

TextChatResponseFormat

The chat response format parameters.

type

Required *

string

Used to enable JSON mode, which guarantees the message the model generates is valid JSON.

Important: when using JSON mode, you must also instruct the model to produce JSON yourself via a system or user message. Without this, the model may generate an unending stream of whitespace until the generation reaches the token limit, resulting in a long-running and seemingly "stuck" request. Also note that the message content may be partially cut off if `finish_reason="length"`, which indicates the generation exceeded `max_tokens` or the conversation exceeded the max context length.

• `text` : Will not create a JSON structured output.
• `json_object` : User will get a JSON structured output, but the schema is not guranteed.
• `json_schema` : User needs to declare the `json_schema` object to provide the desired JSON schema in order to get a matched response.

json_schema

object

User-defined JSON schema object with optional parameters to enforce structured output.

Important: this field will only applied when `response_format`.`type` is set to `json_schema`.

seed

integer

Random number generator seed to use in sampling mode for experimental repeatability.

stop

array

Stop sequences are one or more strings which will cause the text generation to stop if/when they are produced as part of the output. Stop sequences encountered prior to the minimum number of tokens being generated will be ignored.

stream

boolean

If set to true, this operation will return the output tokens as a stream of events, and usage is always included in the response.

temperature

number

What sampling temperature to use,. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.

We generally recommend altering this or `top_p` but not both.

top_p

number

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

We generally recommend altering this or `temperature` but not both.

time_limit

integer

Time limit in milliseconds - if not completed within this time, generation will stop. The text generated so far will be returned along with the `TIME_LIMIT`` stop reason. Depending on the users plan, and on the model being used, there may be an enforced maximum time limit.

Response

200 - Successful operation

id

string

A unique identifier for the chat completion.

model_id

string

The model used for the chat completion.

model

string

This field is a duplicate of `model_id` and is provided in order to provide better compatibility with other APIs.

model_version

string

The model version (using semantic versioning) if set.

created

integer

The Unix timestamp (in seconds) of when the chat completion was created.

created_at

string

The time when the response was created in ISO 8601 format.

usage

TextChatUsage

Usage statistics for the completion request.

moderations

ModerationResults

The result of any detected moderations.

choices

array

A list of chat completion choices. Can be more than one if `n` is greater than 1.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Infer text event stream

POST /ml/v1/text/chat_stream

Infer the next tokens for a given deployed model with a set of PARAMS3. This operation will return the output tokens as a stream of events.

This operation will set `stream` in the request body to true and return the output tokens as a stream of events.

You can also use /v1/chat/completions with `stream` option set to true to achieve the same result.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-chat.send

Request

Request Body

TextChatStreamRequest

From a given prompt, infer the next tokens in a server-sent events (SSE) stream.

model_id

Required *

string

The model to use for the chat completion.

Please refer to the list of models.

model

string

This is an alias for `model_id` and can be used instead of `model_id`.

The field `model_id` is marked as mandatory but it can be replaced by the `model` field for compatibility with other APIs.

messages

Required *

array

The messages for this chat session.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

moderations

Moderations

Properties that control the moderations, for usages such as `Hate and profanity` (HAP) and `Personal identifiable information` (PII) filtering. This list can be extended with new types of moderations.

hap

object

The properties for the moderation. Each type of moderation may have additional properties that are specific to that moderation.

pii

object

The properties for the moderation. Each type of moderation may have additional properties that are specific to that moderation.

granite_guardian

object

The properties specific to Granite Guardian. This detector is in beta and may change.

input_ranges

array

If set, then only these ranges will be applied to the moderations. This is useful in the case that certain parts of the input text have already been checked.

tools

array

Tool functions that can be called with the response.

tool_choice_option

string

Specify either `tool_choice_option` to allow the model to pick or `tool_choice` to force the model to call a tool.

Using `auto` means the model can pick between generating a message or calling one or more tools. Default is `auto`.

Using `none` means the model will not call any tool and instead generates a message.

Using `required` means the model must call one or more tools.

tool_choice

TextChatToolChoiceTool

Specifying a particular tool via `{"type": "function", "function": {"name": "my_function"}}` forces the model to call that tool. Specify either `tool_choice_option` to allow the model to pick or `tool_choice` to force the model to call a tool.

type

Required *

string

The tool type.

function

Required *

object

The named function.

guided_choice

array

If specified, the output will be exactly one of the choices.

guided_regex

string

If specified, the output will follow the regex pattern.

guided_grammar

string

If specified, the output will follow the context free grammar.

guided_json

object

If specified, the output will follow the JSON schema. See the JSON Schema reference for documentation about the format.

chat_template_kwargs

object

Additional kwargs to pass to the chat template, described as a JSON Schema object. See the JSON Schema reference for documentation about the format.

frequency_penalty

number

Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

include_reasoning

boolean

Whether to include `reasoning_content` in the response. Default is `true`.

logit_bias

object

Increasing or decreasing probability of tokens being selected during generation; a positive bias makes a token more likely to appear, while a negative bias makes it less likely.

logprobs

boolean

Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.

top_logprobs

integer

An integer specifying the number of most likely tokens to return at each token position, each with an associated log probability. The option `logprobs` must be set to `true` if this parameter is used.

max_completion_tokens

integer

The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. Set to 0 for the model's configured max generated tokens.

max_tokens

integer

The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. Set to 0 for the model's configured max generated tokens.

This value is now deprecated in favor of `max_completion_tokens`. If specified together with `max_completion_tokens`, `max_tokens` will be ignored.

n

integer

How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Keep n as 1 to minimize costs.

presence_penalty

number

Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

reasoning_effort

string

A lower reasoning effort can result in faster responses, fewer tokens used, and shorter `reasoning_content` in the responses. Supported values are `low`, `medium`, and `high`.

response_format

TextChatResponseFormat

The chat response format parameters.

type

Required *

string

Used to enable JSON mode, which guarantees the message the model generates is valid JSON.

Important: when using JSON mode, you must also instruct the model to produce JSON yourself via a system or user message. Without this, the model may generate an unending stream of whitespace until the generation reaches the token limit, resulting in a long-running and seemingly "stuck" request. Also note that the message content may be partially cut off if `finish_reason="length"`, which indicates the generation exceeded `max_tokens` or the conversation exceeded the max context length.

• `text` : Will not create a JSON structured output.
• `json_object` : User will get a JSON structured output, but the schema is not guranteed.
• `json_schema` : User needs to declare the `json_schema` object to provide the desired JSON schema in order to get a matched response.

json_schema

object

User-defined JSON schema object with optional parameters to enforce structured output.

Important: this field will only applied when `response_format`.`type` is set to `json_schema`.

seed

integer

Random number generator seed to use in sampling mode for experimental repeatability.

stop

array

Stop sequences are one or more strings which will cause the text generation to stop if/when they are produced as part of the output. Stop sequences encountered prior to the minimum number of tokens being generated will be ignored.

stream

boolean

If set to true, this operation will return the output tokens as a stream of events, and usage is always included in the response.

temperature

number

What sampling temperature to use,. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.

We generally recommend altering this or `top_p` but not both.

top_p

number

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

We generally recommend altering this or `temperature` but not both.

time_limit

integer

Time limit in milliseconds - if not completed within this time, generation will stop. The text generated so far will be returned along with the `TIME_LIMIT`` stop reason. Depending on the users plan, and on the model being used, there may be an enforced maximum time limit.

Response

200 - Successful operation (`Content-Type: text/event-stream`).

400 -

401 -

403 -

404 -

Create a new AI service

POST /ml/v4/ai_services

Create a new AI service with the given payload. A AI service is some code that can be deployed as a deployment.

Auditing

Calling this method generates the following auditing event.

• pm-20.ai_service.create

Request

Request Body

AIServiceRequest

Payload for creating the AI service. Either `space_id` or `project_id` has to be provided and is mandatory.

space_id

Required *

string

The space that contains the resource.

name

Required *

string

The name of the resource.

description

string

A description of the resource.

tags

Tags

A list of tags for this resource.

software_spec

Required *

SoftwareSpecRel

A software specification.

id

string

The id of the software specification.

rev

string

The revision of the software specification.

name

string

The name of the software specification.

code_type

string

The type that allows the deployment service to know how to setup the code during deployment.

documentation

AIServiceDocumentation

The documentation of the AI service request body and response body.

request

object

The schema for a given content type. Each property defines the content type and the sub-object is the JSON schema that describes the content.

response

object

The schema for a given content type. Each property defines the content type and the sub-object is the JSON schema that describes the content.

init

object

A JSON schema, see the JSON Schema reference for documentation about the format.

functions

object

The documentation for which endpoints are supported. If any of the fields do not exist then they are assumed to be `true`.

If the deployment only supports batch jobs then the `functions` object would be declared like this:

```json
"functions": {
"generate": false,
"generate_stream": false
}
```

If the deployment only supports online scoring, with streaming support as well, then the `functions` object would be declared like this:

```json
"functions": {
"generate_batch": false
}
```

custom

Custom

User defined properties specified as key-value pairs.

tooling

Tooling

User defined properties specified as key-value pairs, which is propagated to the deployment.

Response

201 - AI service created

metadata

ResourceMeta

Common metadata for a simple resource.

entity

AIServiceEntity

The details of the AI service to be created.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Retrieve the AI services

GET /ml/v4/ai_services

Retrieve the AI services for the specified space or project.

Auditing

Calling this method generates the following auditing event.

• pm-20.ai_service.list

Request

Path Parameters

Query Parameters

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

start

string

Token required for token-based pagination. This token cannot be determined by end user. It is generated by the service and it is set in the href available in the `next` field.

limit

integer

How many resources should be returned. By default limit is 100. Max limit allowed is 200.

tag.value

string

Return only the resources with the given tag values, separated by `or` or `and` to support multiple tags.

search

string

Returns only resources that match this search string. The path to the field must be the complete path to the field, and this field must be one of the indexed fields for this resource type. Note that the search string must be URL encoded.

These are the fields that can be searched in the `metadata`:

• `/metadata/name`

Note that `tags` are filtered using the `tag` query parameter and the `tag` query parameter takes precedence over the `search` query parameter.

The metadata fields, on all assets, can be searched like this:

• `/metadata/name=asset2` -> `search=%2Fmetadata%2Fname%3Dasset2`

These are the fields that can be searched in the `entity` and that depend on the asset type: - `model` - `/entity/type` - `/entity/software_spec.id` - `function` - `/entity/software_spec.id` - `ai_service` - `/entity/software_spec.id`

The entity fields can be searched like this:

`/entity.type=tensorflow_2.14` -> `search=%2Fentity%2Ftype%3Dtensorflow_2.14`

Response

200 - OK

total_count

integer

The total number of resources. Computed explicitly only when `total_count=true` query parameter is present. This is in order to avoid performance penalties.

limit

integer

The number of items to return in each page.

first

PaginationFirst

The reference to the first item in the current page.

next

PaginationNext

A reference to the first item of the next page, if any.

resources

array

A list of AI services.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Retrieve the AI service

GET /ml/v4/ai_services/{id}

Retrieve the AI service with the specified identifier. If `rev` query parameter is provided, `rev=latest` will fetch the latest revision. A call with `rev={revision_number}` will fetch the given revision_number record. Either `space_id` or `project_id` has to be provided and is mandatory.

Auditing

Calling this method generates the following auditing event.

• pm-20.ai_service.read

Request

Path Parameters

Query Parameters

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

rev

string

The revision number of the resource.

Response

200 - OK

metadata

ResourceMeta

Common metadata for a simple resource.

entity

AIServiceEntity

The details of the AI service to be created.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Update the AI service

PATCH /ml/v4/ai_services/{id}

Update the AI service with the provided patch data. The following fields can be patched:

• `/tags`
• `/name`
• `/description`
• `/custom`

Auditing

Calling this method generates the following auditing event.

• pm-20.ai_service.update

Request

Request Body

JsonPatch

Input For Patch. This is the patch body which corresponds to the JavaScript Object Notation (JSON) Patch standard (RFC 6902).

Path Parameters

Query Parameters

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

Response

200 - AI service has been patched successfully

metadata

ResourceMeta

Common metadata for a simple resource.

entity

AIServiceEntity

The details of the AI service to be created.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Delete the AI service

DELETE /ml/v4/ai_services/{id}

Delete the AI service with the specified identifier. This will delete all revisions of this flow as well. For each revision all attachments will also be deleted.

Auditing

Calling this method generates the following auditing event.

• pm-20.ai_service.delete

Request

Path Parameters

Query Parameters

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

Response

204 - AI service deleted

400 -

401 -

403 -

404 -

Upload the AI service code

PUT /ml/v4/ai_services/{id}/code

Upload the flow code. AI services expect a zip file that contains the code files that make up the flow.

Auditing

Calling this method generates the following auditing event.

• pm-20.ai_service.add

Request

Request Body

Path Parameters

Query Parameters

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

Response

201 - AI service code uploaded

attachment_id

string

The content id for the attachment.

400 -

401 -

403 -

404 -

Download the AI service code

GET /ml/v4/ai_services/{id}/code

Download the AI service code. It is possible to download the `code` for a given revision of the `flow`. AI services expect a zip file that contains the code files that make up the flow.

Auditing

Calling this method generates the following auditing event.

• pm-20.ai_service.read

Request

Path Parameters

Query Parameters

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

rev

string

The revision number of the resource.

Response

200 -

400 -

401 -

403 -

404 -

Create a new AI service revision

POST /ml/v4/ai_services/{id}/revisions

Create a new AI service revision. The current metadata and content for `id` will be taken and a new revision created. Either `space_id` or `project_id` has to be provided and is mandatory.

Auditing

Calling this method generates the following auditing event.

• pm-20.ai_service.create

Request

Request Body

RevisionEntitySpaceProjectRequest

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

commit_message

string

An optional commit message for the revision.

Response

201 - AI service revision created

metadata

ResourceMeta

Common metadata for a simple resource.

entity

AIServiceEntity

The details of the AI service to be created.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Retrieve the AI service revisions

GET /ml/v4/ai_services/{id}/revisions

Retrieve the AI service revisions.

Auditing

Calling this method generates the following auditing event.

• pm-20.ai_service.list

Request

Path Parameters

Query Parameters

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

start

string

Token required for token-based pagination. This token cannot be determined by end user. It is generated by the service and it is set in the href available in the `next` field.

limit

integer

How many resources should be returned. By default limit is 100. Max limit allowed is 200.

Response

200 - AI service revisions

total_count

integer

The total number of resources. Computed explicitly only when `total_count=true` query parameter is present. This is in order to avoid performance penalties.

limit

integer

The number of items to return in each page.

first

PaginationFirst

The reference to the first item in the current page.

next

PaginationNext

A reference to the first item of the next page, if any.

resources

array

A list of AI services.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

POST /ml/v1/text/detection

This operation is used mainly for Hate And Profanity (`HAP`) and Personal Identifiable Information (`PII`) filtering.

This is a detection-only end-point. It supports natural language input and output and returns the result of the detection. It can be configured for HAP, PII, or any combination with other available detectors.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-detection-content.send

Request

Request Body

TextDetectionContentRequest

From a content, detect text.

input

Required *

string

The text to be examined.

detectors

Required *

TextDetectionContentDetectors

The detectors to use, these can be IBM provided `HAP` or `PII` detectors or a custom content detector.

hap

object

The definition of a `Hate and profanity` (HAP) text detector.

pii

object

The definition of a `Personal identifiable information` (PII) text detector.

granite_guardian

object

The definition of a `Granite Guardian` text detector. This detector is in beta and may change.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

Response

200 - Successful operation

detections

array

The text that was detected.

400 -

401 -

403 -

404 -

POST /ml/v1/text/detection/context

This operation supports `context relevance` and `faithfulness` (or `groundedness`).

The `input` is analyzed, along with the context information, and the model will return any detections that it found.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-detection-context.send

Request

Request Body

TextDetectionContextRequest

From a content, detect text using a detection model.

input

Required *

string

The text to be examined.

detectors

Required *

TextDetectionDetectors

The detectors to use, this is a map of `detector-name` with a map of optional key/value pairs.

context_type

Required *

string

The type of the context.

context

Required *

object

Context documents.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

Response

200 - Successful operation

detections

array

The text that was detected.

400 -

401 -

403 -

404 -

POST /ml/v1/text/detection/generated

This operation supports `answer relevance`.

The `prompt` is analyzed, along with the `generated text`, and the model will return any detections that it found.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-detection-generated.send

Request

Request Body

TextDetectionGeneratedRequest

From a content, detect text using a detection model.

prompt

Required *

string

The text prompt.

generated_text

Required *

string

The generated text.

detectors

Required *

TextDetectionDetectors

The detectors to use, this is a map of `detector-name` with a map of optional key/value pairs.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

Response

200 - Successful operation

detections

array

The text that was detected.

400 -

401 -

403 -

404 -

Time series forecast

POST /ml/v1/time_series/forecast

Generate forecasts, or predictions for future time points, given historical time series data.

Auditing

Calling this method generates the following auditing event.

• pm-20.time-series-forecast.send

Request

Request Body

TSForecastRequest

The forecast request.

project_id

string

The project id of the resource.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

model_id

Required *

string

The model to be used for generating a forecast. You can get the list of models by using Foundation Model Specs with `filters=function_time_series_forecast`.

parameters

TSCommonForecastParameters

The parameters for the forecast request.

prediction_length

integer

The prediction length for the forecast. The service will return this many periods beyond the last timestamp in the inference data payload. If specified, `prediction_length` must be an integer >=1 and no more than the model default prediction length. When omitted the model default prediction_length will be used.

data

Required *

object

A payload of data matching `schema`. We assume the following about your data: * All timeseries are of equal length and are uniform in nature (the time difference between two successive rows is constant). This implies that there are no missing rows of data; * The data meet the minimum model-dependent historical context length which can be 512 or more rows per timeseries;

Note that the example payloads shown are for illustration purposes only. An actual payload would necessary be much larger to meet minimum model-specific context lengths.

schema

Required *

TSForecastInputSchema

Contains metadata about your timeseries data input.

timestamp_column

Required *

string

A valid column in the data that should be treated as the timestamp. Although not absolutely necessary, if using calendar dates (simple integer time offsets are also allowed), users should consider using a format such as ISO 8601 that includes a UTC offset (e.g., '2024-10-18T01:09:21.454746+00:00'). This will avoid potential issues such as duplicate dates appearing due to daylight savings change overs. There are many date formats in existence and inferring the correct one can be a challenge so please do consider adhering to ISO 8601.

id_columns

array

Columns that define a unique key for timeseries. This is similar to a compound primary key in a database table.

freq

string

A frequency indicator for the given timestamp_column. See https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#period-aliases for a description of the allowed values. If not provided, we will attempt to infer it from the data.

target_columns

array

An array of column headings which constitute the target variables in the data. These are the data that will be forecasted.

Response

200 - Successful operation

model_id

string

The model used to generate the forecast.

created_at

string

The time when the response was created in ISO 8601 format.

results

array

The list of prediction results. There will be a forecast for each time series in the input data. The `prediction_length` field in the request specifies the number of predictions in the results. The actual number of rows in the results will be equal to the `prediction length` multiplied by the number of unique ids in `id_columns`. The `timestamp_column` field in the request indicates the name of the timestamp column in the results.

input_data_points

integer

The number of input data points (number of rows in `data` * number of input columns in `data`).

output_data_points

integer

The number of forecasted data points (`prediction_length` * number of `target_columns` * number of unique ids in `id_columns`).

400 -

401 -

403 -

404 -

Submit a new Evaluation

POST /ml/v1/text/evaluations

Request

Request Body

EvaluationRequest

name

Required *

string

The name of the job.

description

string

The description of the job.

tags

Tags

A list of tags for this resource.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

model_settings

EvaluationModelConfig

Settings related to evaluated foundation model.

model_id

string

The ID of the pre-installed foundation model. Either `model_id` or `deployment_id` parameter has to be given.

deployment_id

string

The ID of the deployed foundation model. Either `model_id` or `deployment_id` parameter has to be given.

generation_parameters

object

Properties that control the model and response.

task_settings

Required *

EvaluationTaskConfig

Settings related to evaluation job.

tasks

Required *

array

The evaluation tasks to be run.

sample_size

integer

Sample size limit for given tasks.

Response

201 - Job successfully submitted

metadata

AutoAIRAGMetadata

Common metadata for a simple resource.

entity

EvaluationResourceEntity

The evaluation request entity.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 - Invalid request parameters

error

string

Error message

Get list of Evaluations

GET /ml/v1/text/evaluations

Request

Response

200 - OK

total_count

integer

The total number of resources. Computed explicitly only when `total_count=true` query parameter is present. This is in order to avoid performance penalties.

limit

integer

The number of items to return in each page.

first

PaginationFirst

The reference to the first item in the current page.

next

PaginationNext

A reference to the first item of the next page, if any.

resources

array

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Get Evalutation with given ID

GET /ml/v1/text/evaluations/{id}

Request

Response

200 - OK

metadata

object

Common metadata for a simple resource.

entity

EvaluationResourceEntity

The evaluation request entity.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

404 -

Cancel or delete Evaluation with given ID

DELETE /ml/v1/text/evaluations/{id}

Request

Path Parameters

Query Parameters

hard_delete

boolean

If set to `true`, evaluation job gets deleted instead of cancelled.

Response

204 - No Content

404 -

POST /v2/synthetic_data/generation/unstructured

Create a synthetic unstructured data generation job.

This API is deprecated.

Request

Request Body

SDGUnstructuredGenerationRequest-v2

name

Required *

string

The name of the resource.

description

string

A description of the resource.

project_id

Required *

string

The project that contains the resource.

configuration

object

pipeline

Required *

string

The pipeline type.

num_outputs_to_generate

integer

How many QnA pairs to generate.

generator

object

Data builder configuration.

validators

array

seed_data_reference

Required *

object

A reference to data.

results_reference

Required *

object

A reference to data.

knowledge_base_references

array

overwrite_output_file

boolean

Flag to allow overwriting existing output file. If false or not set, job creation will fail if output file already exists.

Response

201 - The synthetic data generation job has been created.

metadata

object

A reference to a resource.

entity

SDGUnstructuredGenerationConfig

The details needed to create unstructured synthetic data generation job.

400 -

401 -

403 -

See available GPUs

GET /ml/v1/gpus

See all available GPUs on the cluster.

Auditing

Calling this method generates the following auditing event.

• pm-20.gpus.list

Request

Path Parameters

Query Parameters

start

string

Token required for token-based pagination. This token cannot be determined by end user. It is generated by the service and it is set in the href available in the `next` field.

limit

integer

How many resources should be returned. By default limit is 100. Max limit allowed is 200.

gpu_id

string

The GPU id for filtering.

gpu_type

string

The GPU type for filtering.

Response

200 - OK

total_count

integer

Computed explicitly only when `total_count=true` query parameter is present. This is in order to avoid performance penalties.

limit

integer

The number of items to return in each page.

first

PaginationFirst

The reference to the first item in the current page.

next

PaginationNext

A reference to the first item of the next page, if any.

gpu_configurations

array

A list of available GPUs.

400 -

401 -

403 -

404 -

Start a text classification request

POST /ml/v1/text/classifications

Start a request to classify text from a document or an image (using OCR).

Auditing

Calling this method generates the following auditing event.

• pm-20.text-classification.send

Request

Request Body

TextClassificationRequest

The input for the text classification request.

document_reference

Required *

TextClassificationDataReference

A reference to data.

type

Required *

string

The data source type.

connection

object

Contains a set of location fields specific to each data source.

location

object

Contains a set of fields specific to each connection.

parameters

TextClassificationParameters

The parameters for the text extraction.

ocr_mode

string

If OCR should be used when processing a document. An empty value allows the service to select the best option for your processing mode.

• `enabled`: OCR is run on embedded images, OCR is only run if no programmatic text could be extracted from the area.
• `disabled`: OCR is not run, no information is extracted from images or scanned documents.
• `forced`: Document processing will take a picture of the page and run OCR across it, this applies to all documents even purely programmatic ones.

classification_mode

string

The classification mode. The value `exact` gives the exact schema name the the document is classified to. The option `binary`` only gives whether the document is classified to a known schema or not.

auto_rotation_correction

boolean

Should the service attempt to fix a rotated page or image.

languages

array

Set of languages to be expected in the document. The language codes follow `ISO 639` where possible. See the documentation for the currently supported languages.

semantic_config

object

Properties related to semantic config.

custom

Custom

User defined properties specified as key-value pairs.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

Response

201 - Created. The `Content-Location` header will contain the URI reference to the created resource.

metadata

TextExtractionMetadata

Common metadata for a simple resource.

entity

object

The parameters for the text classification.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

GET /ml/v1/text/classifications

Retrieve the list of text classification requests for the specified space or project.

This operation does not save the history, any requests that were deleted or purged will not appear in this list.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-classification.list

Request

Path Parameters

Query Parameters

version

Required *

string

The version date for the API of the form `YYYY-MM-DD`.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

start

string

Token required for token-based pagination. This token cannot be determined by end user. It is generated by the service and it is set in the href available in the `next` field.

limit

integer

How many resources should be returned. By default limit is 100. Max limit allowed is 200.

Response

200 - OK.

total_count

integer

The total number of resources. Computed explicitly only when `total_count=true` query parameter is present. This is in order to avoid performance penalties.

limit

integer

The number of items to return in each page.

first

PaginationFirst

The reference to the first item in the current page.

next

PaginationNext

A reference to the first item of the next page, if any.

resources

array

A list of resources.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

GET /ml/v1/text/classifications/{id}

Retrieve the text classification request with the specified identifier.

Note that there is a retention period of 2 days. If this retention period is exceeded then the request will be deleted and the results no longer available. In this case this operation will return `404`.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-classification.get

Request

Response

200 - OK.

metadata

TextExtractionMetadata

Common metadata for a simple resource.

entity

object

The parameters for the text classification.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Delete the request

DELETE /ml/v1/text/classifications/{id}

Cancel the specified text classification request and delete any associated results.

Auditing

Calling this method generates the following auditing event.

• pm-20.text-classification.delete

Request

Path Parameters

Query Parameters

hard_delete

boolean

Set to true in order to also delete the job or request metadata.

Response

204 - Request deleted.

400 -

401 -

403 -

404 -

Start a create schema request

POST /ml/v1/text/schemas/create

Start a request to create the custom schema for text extraction

Auditing

Calling this method generates the following auditing event.

• pm-20.create-schema.send

Request

Request Body

CreateSchemaRequest

The input for the create schema request.

document_reference

Required *

TextExtractionDataReference

A reference to data.

type

Required *

string

The data source type.

connection

object

Contains a set of location fields specific to each data source.

location

object

Contains a set of fields specific to each connection.

parameters

CreateSchemaParameters

The parameters for the create schema.

mode

string

Which processing mode to use for this request.

ocr_mode

string

If OCR should be used when processing a document. An empty value allows the service to select the best option for your processing mode.

• `enabled`: OCR is run on embedded images, OCR is only run if no programmatic text could be extracted from the area.
• `disabled`: OCR is not run, no information is extracted from images or scanned documents.
• `forced`: WDU will take a picture of the page and run OCR across it, this applies to all documents even purely programmatic ones.

auto_rotation_correction

boolean

Should the service attempt to fix a rotated page or image.

languages

array

Set of languages to be expected in the document. The language codes follow `ISO 639` where possible. See the documentation for the currently supported languages.

additional_prompt_instructions

string

Additional instructions to guide schema creation.

enable_grounding

boolean

If we should return grounding data with examples of each field.

max_pages_to_process

integer

How many pages we should create a schema for.

semantic_config

object

Properties related to semantic config.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

Response

201 - Created. The `Content-Location` header will contain the URI reference to the created resource.

metadata

TextExtractionMetadata

Common metadata for a simple resource.

entity

object

The parameters to create schema.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

GET /ml/v1/text/schemas/create

Retrieve the list of create schema requests for the specified space or project.

This operation does not save the history, any requests that were deleted or purged will not appear in this list.

Auditing

Calling this method generates the following auditing event.

• pm-20.create-schema.list

Request

Path Parameters

Query Parameters

version

Required *

string

The version date for the API of the form `YYYY-MM-DD`.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

start

string

Token required for token-based pagination. This token cannot be determined by end user. It is generated by the service and it is set in the href available in the `next` field.

limit

integer

How many resources should be returned. By default limit is 100. Max limit allowed is 200.

Response

200 - OK.

total_count

integer

The total number of resources. Computed explicitly only when `total_count=true` query parameter is present. This is in order to avoid performance penalties.

limit

integer

The number of items to return in each page.

first

PaginationFirst

The reference to the first item in the current page.

next

PaginationNext

A reference to the first item of the next page, if any.

resources

array

A list of resources.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

GET /ml/v1/text/schemas/create/{id}

Retrieve the create schema request with the specified identifier.

Note that there is a retention period of 2 days. If this retention period is exceeded then the request will be deleted and the results no longer available. In this case this operation will return `404`.

Auditing

Calling this method generates the following auditing event.

• pm-20.create-schema.get

Request

Response

200 - OK.

metadata

TextExtractionMetadata

Common metadata for a simple resource.

entity

object

The parameters to create schema.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Delete the request

DELETE /ml/v1/text/schemas/create/{id}

Cancel the specified create schema request and delete any associated results.

Auditing

Calling this method generates the following auditing event.

• pm-20.create-schema.delete

Request

Path Parameters

Query Parameters

hard_delete

boolean

Set to true in order to also delete the job or request metadata.

Response

204 - Request deleted.

400 -

401 -

403 -

404 -

Start a improve schema request

POST /ml/v1/text/schemas/improve

Start a request to create the custom schema for text extraction

Auditing

Calling this method generates the following auditing event.

• pm-20.improve-schema.send

Request

Request Body

ImproveSchemaRequest

The input for the improve schema request.

parameters

Required *

ImproveSchemaParameters

The parameters to improve schema.

schema

Required *

object

A custom input schema given by the user.

semantic_config

object

Properties related to semantic config.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

Response

201 - Created. The `Content-Location` header will contain the URI reference to the created resource.

metadata

TextExtractionMetadata

Common metadata for a simple resource.

entity

object

The parameters to improve schema.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

GET /ml/v1/text/schemas/improve

Retrieve the list of improve schema requests for the specified space or project.

This operation does not save the history, any requests that were deleted or purged will not appear in this list.

Auditing

Calling this method generates the following auditing event.

• pm-20.improve-schema.list

Request

Path Parameters

Query Parameters

version

Required *

string

The version date for the API of the form `YYYY-MM-DD`.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

start

string

Token required for token-based pagination. This token cannot be determined by end user. It is generated by the service and it is set in the href available in the `next` field.

limit

integer

How many resources should be returned. By default limit is 100. Max limit allowed is 200.

Response

200 - OK.

total_count

integer

The total number of resources. Computed explicitly only when `total_count=true` query parameter is present. This is in order to avoid performance penalties.

limit

integer

The number of items to return in each page.

first

PaginationFirst

The reference to the first item in the current page.

next

PaginationNext

A reference to the first item of the next page, if any.

resources

array

A list of resources.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

GET /ml/v1/text/schemas/improve/{id}

Retrieve the improve schema request with the specified identifier.

Note that there is a retention period of 2 days. If this retention period is exceeded then the request will be deleted and the results no longer available. In this case this operation will return `404`.

Auditing

Calling this method generates the following auditing event.

• pm-20.improve-schema.get

Request

Response

200 - OK.

metadata

TextExtractionMetadata

Common metadata for a simple resource.

entity

object

The parameters to improve schema.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Delete the request

DELETE /ml/v1/text/schemas/improve/{id}

Cancel the specified improve schema request and delete any associated results.

Auditing

Calling this method generates the following auditing event.

• pm-20.improve-schema.delete

Request

Path Parameters

Query Parameters

hard_delete

boolean

Set to true in order to also delete the job or request metadata.

Response

204 - Request deleted.

400 -

401 -

403 -

404 -

Start a merge schema request

POST /ml/v1/text/schemas/merge

Start a request to merge a list of semantically similar custom schemas for text extraction into a single schema.

Auditing

Calling this method generates the following auditing event.

• pm-20.improve-schema.send

Request

Request Body

MergeSchemaRequest

The input for the merge schema request.

parameters

Required *

MergeSchemaParameters

The parameters to merge schema.

schemas

Required *

array

A list of input schemas.

semantic_config

object

Properties related to semantic config.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

Response

201 - Created. The `Content-Location` header will contain the URI reference to the created resource.

metadata

TextExtractionMetadata

Common metadata for a simple resource.

entity

object

The parameters to merge schema.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

GET /ml/v1/text/schemas/merge

Retrieve the list of merge schema requests for the specified space or project.

This operation does not save the history, any requests that were deleted or purged will not appear in this list.

Auditing

Calling this method generates the following auditing event.

• pm-20.merge-schema.list

Request

Path Parameters

Query Parameters

version

Required *

string

The version date for the API of the form `YYYY-MM-DD`.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

start

string

Token required for token-based pagination. This token cannot be determined by end user. It is generated by the service and it is set in the href available in the `next` field.

limit

integer

How many resources should be returned. By default limit is 100. Max limit allowed is 200.

Response

200 - OK.

total_count

integer

The total number of resources. Computed explicitly only when `total_count=true` query parameter is present. This is in order to avoid performance penalties.

limit

integer

The number of items to return in each page.

first

PaginationFirst

The reference to the first item in the current page.

next

PaginationNext

A reference to the first item of the next page, if any.

resources

array

A list of resources.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

GET /ml/v1/text/schemas/merge/{id}

Retrieve the merge schema request with the specified identifier.

Note that there is a retention period of 2 days. If this retention period is exceeded then the request will be deleted and the results no longer available. In this case this operation will return `404`.

Auditing

Calling this method generates the following auditing event.

• pm-20.merge-schema.get

Request

Response

200 - OK.

metadata

TextExtractionMetadata

Common metadata for a simple resource.

entity

object

The parameters to merge schema.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Delete the request

DELETE /ml/v1/text/schemas/merge/{id}

Cancel the specified merge schema request and delete any associated results.

Auditing

Calling this method generates the following auditing event.

• pm-20.merge-schema.delete

Request

Path Parameters

Query Parameters

hard_delete

boolean

Set to true in order to also delete the job or request metadata.

Response

204 - Request deleted.

400 -

401 -

403 -

404 -

Start a cluster schema request

POST /ml/v1/text/schemas/cluster

Start a request to cluster a list of custom schemas into semantically similar groups.

Auditing

Calling this method generates the following auditing event.

• pm-20.cluster-schema.send

Request

Request Body

ClusterSchemaRequest

The input for the cluster schema request.

parameters

Required *

ClusterSchemaParameters

The parameters to cluster schemas.

schemas

Required *

array

A list of document schemas.

semantic_config

object

Properties related to semantic config.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` has to be given.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` has to be given.

Response

201 - Created. The `Content-Location` header will contain the URI reference to the created resource.

metadata

TextExtractionMetadata

Common metadata for a simple resource.

entity

object

The parameters to cluster schema.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

GET /ml/v1/text/schemas/cluster

Retrieve the list of cluster schema requests for the specified space or project.

This operation does not save the history, any requests that were deleted or purged will not appear in this list.

Auditing

Calling this method generates the following auditing event.

• pm-20.cluster-schema.list

Request

Path Parameters

Query Parameters

version

Required *

string

The version date for the API of the form `YYYY-MM-DD`.

space_id

string

The space that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

project_id

string

The project that contains the resource. Either `space_id` or `project_id` query parameter has to be given.

start

string

Token required for token-based pagination. This token cannot be determined by end user. It is generated by the service and it is set in the href available in the `next` field.

limit

integer

How many resources should be returned. By default limit is 100. Max limit allowed is 200.

Response

200 - OK.

total_count

integer

The total number of resources. Computed explicitly only when `total_count=true` query parameter is present. This is in order to avoid performance penalties.

limit

integer

The number of items to return in each page.

first

PaginationFirst

The reference to the first item in the current page.

next

PaginationNext

A reference to the first item of the next page, if any.

resources

array

A list of resources.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

GET /ml/v1/text/schemas/cluster/{id}

Retrieve the cluster schema request with the specified identifier.

Note that there is a retention period of 2 days. If this retention period is exceeded then the request will be deleted and the results no longer available. In this case this operation will return `404`.

Auditing

Calling this method generates the following auditing event.

• pm-20.cluster-schema.get

Request

Response

200 - OK.

metadata

TextExtractionMetadata

Common metadata for a simple resource.

entity

object

The parameters to cluster schema.

system

SystemDetails

Optional details coming from the service and related to the API call or the associated resource.

400 -

401 -

403 -

404 -

Delete the request

DELETE /ml/v1/text/schemas/cluster/{id}

Cancel the specified cluster schema request and delete any associated results.

Auditing

Calling this method generates the following auditing event.

• pm-20.cluster-schema.delete

Request

Path Parameters

Query Parameters

hard_delete

boolean

Set to true in order to also delete the job or request metadata.

Response

204 - Request deleted.

400 -

401 -

403 -

404 -

Create Speech

POST /ml/gateway/v1/audio/speech

Generates speech from text using the specified model.

Request

Request Body

CreateSpeechRequest

Create Speech Request

input

Required *

string

The text to generate audio for. The maximum length is 4096 characters.

instructions

string

Control the voice of your generated audio with additional instructions. For OpenAI, does not work with `tts-1` or `tts-1-hd`.

model

Required *

string

The ID or alias of the model to use for speech generation. For OpenAI, one of the available TTS models: `tts-1`, `tts-1-hd` or `gpt-4o-mini-tts`.

response_format

string

Specify the format of the audio data being generated. For OpenAI, supported formats are `"mp3"`, `"opus"`, `"aac"`, `"flac"`, `"wav"`, and `"pcm"`.

speed

number

The speed of the generated audio. Select a value from `0.25` to `4.0`. `1.0` is the default.

stream_format

string

The format to stream the audio in. Supported formats are `"sse"` and `"audio"`. `"sse"` is not supported for OpenAI's `tts-1` or `tts-1-hd` models.

voice

Required *

string

The voice to use when generating the audio. For OpenAI, supported voices are `"alloy"`, `"ash"`, `"ballad"`, `"coral"`, `"echo"`, `"fable"`, `"onyx"`, `"nova"`, `"sage"`, `"shimmer"`, and `"verse"`. Previews of supported voices for OpenAI are available in their Text to speech guide.

Response

200 - Successful Speech Generation

400 -

401 -

500 -

Create Chat Completions

POST /ml/gateway/v1/chat/completions

Generate a chat completion based on the provided messages and parameters using the provided model.

Request

Request Body

CreateChatsRequest

Chat Completion Request

model

Required *

string

The ID or alias of the model to forward the chat request to.

messages

Required *

array

A list of messages comprising the chat conversation so far. Depending on the model you use, different message types (modalities) are supported, like `"text"`, `"images"`, and `"audio"`.

audio

object

Parameters for audio output. Only required when audio output is requested with modalities: `["audio"]`.

See: OpenAI's Audio Guide for more information.

frequency_penalty

number

A number between `-2.0` and `2.0`. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

function_call

ChatsFunctionCall

Controls which (if any) function is called by the model.

• `"none"` means the model will not call a function and instead generates a message.
• `"auto"` means the model can pick between generating a message or calling a function.
• Specifying a particular function via `{"name": "my_function"}` forces the model to call that function.

`"none"` is the default when no functions are present. `"auto"` is the default if functions are present.

Deprecated: `function_call` has been deprecated by OpenAI in favor of `tool_choice`.

functions

object

A list of functions the model may generate JSON inputs for.

Deprecated: `functions` has been deprecated by OpenAI in favor of `tools`.

logit_bias

object

Modifies the likelihood of specified tokens appearing in the completion. Accepts a JSON object that maps tokens (specified by their token ID in the tokenizer) to an associated bias value from `-100` to `100`. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between `-1` and `1` should decrease or increase likelihood of selection; values like `-100` or `100` should result in a ban or exclusive selection of the relevant token.

logprobs

boolean

Indicates whether to return log probabilities of the output tokens or not. If `true`, returns the log probabilities of each output token returned in the content of message.

max_completion_tokens

integer

Specifies an upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens.

max_tokens

integer

Specifies a maximum number of tokens that can be generated in the chat completion. This value can be used to control costs for text generated via API.

Deprecated: `max_tokens` has been deprecated by OpenAI in favor of `max_completion_tokens`, and is not compatible with `o1` series models.

metadata

ChatsMetadata

Contains developer-defined tags and values used for filtering completions.

modalities

array

Specifies the output types that you would like the model to generate for this request. Most models are capable of generating text, which is the default (`["text"]`). Some models can generate audio. For OpenAI, the `gpt-4o-audio-preview` model can be used to generate audio. To request that this model generate both text and audio responses, you can use `["text", "audio"]`.

n

integer

Specifies how many chat completion choices to generate for each input message.

Note: you will be charged based on the number of generated tokens across all choices, keep `"n"` set to `1` to minimize costs.

parallel_tool_calls

boolean

Specifies whether to enable parallel function calling during tool use.

prediction

ChatsPrediction

The configuration for a Predicted Output, which can greatly improve response times when large parts of the model response are known ahead of time. This is most common when you are regenerating a file with only minor changes to most of the content.

type

Required *

string

Type of predicted content you want to provide, should always be `"content"`.

content

Required *

ChatsPredictionContent

Content that should be matched when generating a model response. If generated tokens would match this content, the entire model response can be returned much more quickly.

presence_penalty

number

A number between `-2.0` and `2.0`. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

reasoning_effort

string

Constrains effort on reasoning for reasoning models. For OpenAI, currently supported by `o1` models only. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response.

response_format

ChatsResponseFormat

An object specifying the format that the model must output.

• Setting to `{ "type": "json_schema", "json_schema": {...} }` enables Structured Outputs which ensures the model will match your supplied JSON schema.
• Setting to `{ "type": "json_object" }` enables JSON mode, which ensures the message the model generates is valid JSON.

Important: when using JSON mode, you must also instruct the model to produce JSON yourself via a system or user message. Without this, the model may generate an unending stream of whitespace until the generation reaches the token limit, resulting in a long-running and seemingly "stuck" request. Also note that the message content may be partially cut off if `"finish_reason"` is set to `"length"`, which indicates the generation exceeded `max_tokens` or the conversation exceeded the max context length.

seed

integer

The seed for the model request. For OpenAI, this feature is in Beta. If specified, OpenAI's system will make a best effort attempt to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed, and you should refer to the `system_fingerprint` response parameter to monitor changes in the backend.

service_tier

string

The service tier used for processing a request.

stop

ChatsStop

Specifies up to 4 sequences where the API will stop generating further tokens.

store

boolean

Indicates whether to store the output of this chat completion request for use in OpenAI's model distillation or evals products.

stream

boolean

Indicates whether to stream the model response to the user. If set, partial message deltas will be sent, like in ChatGPT. Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: `[DONE]` message.

stream_options

StreamOptions

Options for streaming response. Only set this when you set `stream` to `true`.

include_usage

boolean

If set, an additional chunk will be streamed before the data: [DONE] message. The usage field on this chunk shows the token usage statistics for the entire request, and the choices field will always be an empty array. All other chunks will also include a usage field, but with a null value.

temperature

number

Specifies what sampling temperature to use. Higher values like `0.8` will make the output more random, while lower values like `0.2` will make it more focused and deterministic.

Note: OpenAI generally recommends altering this or `top_p` but not both.

tool_choice

ChatsToolChoice

Controls which (if any) tool is called by the model.

• `"none"` means the model will not call any tool and instead generates a message.
• `"auto"` means the model can pick between generating a message or calling one or more tools.
• `"required"` means the model must call one or more tools.
• Specifying a particular tool via `{"type": "function", "function": {"name": "my_function"}}` forces the model to call that tool.

`"none"` is the default when no tools are present. `"auto"` is the default if tools are present.

tools

array

A list of tools the model may call. Currently, only functions are supported as tools. Use this to provide a list of functions the model may generate JSON inputs for. A max of 128 functions are supported.

top_logprobs

integer

An integer between `0` and `20` specifying the number of most likely tokens to return at each token position, each with an associated log probability. LogProbs must be set to `true` if this parameter is used.

top_p

number

An alternative to sampling with `temperature`, called nucleus sampling, where the model considers the results of the tokens with `top_p` probability mass. Example: `0.1` means only the tokens comprising the top 10% probability mass are considered.

Note: OpenAI generally recommends altering this or `temperature` but not both.

user

string

A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse.

router

object

Specifies model routing configurations for the request.

cache

ChatsCache

Contains the caching configuration for a request. Cache is only supported for non-streaming requests.

enabled

boolean

Specifies whether to enable caching for the current request.

filter

object

The filter criteria for caching.

threshold

number

The threshold for caching the request; required if cache is enabled.

Response

201 - Created Successfully

id

string

The unique identifier for the chat completion.

object

string

Object is the response object's type, which should always be `"chat.completion"`.

created

integer

The UNIX timestamp (in seconds) of when the chat completion was created.

model

string

The ID of the model used for the chat completion.

choices

array

A list of chat completion choices. Can be more than one if `n` is greater than `1` in the request.

service_tier

string

The service tier used for processing a request.

system_fingerprint

string

The backend configuration that the model runs with. Can be used in conjunction with the seed request parameter to understand when backend changes have been made that might impact determinism.

usage

Usage

Usage information for a model request.

cached

boolean

Indicates whether the request was cached.

400 -

401 -

500 -

Create Text Completions

POST /ml/gateway/v1/completions

Generate a text completion based on the provided prompt and parameters using the provided model.

Request

Request Body

CreateCompletionsRequest

Completion Request

model

Required *

string

The ID or alias of the model to use.

prompt

Required *

string

The prompt(s) to generate completions for, encoded as a string, array of strings, array of tokens, or array of token arrays.

Note: `<|endoftext|>` is the document separator that the model sees during training, so if a prompt is not specified the model will generate as if from the beginning of a new document.

best_of

integer

Generates `best_of` number of completions server-side and returns the "best" (the one with the highest log probability per token). Results cannot be streamed. When used with `n`, `best_of` controls the number of candidate completions and `n` specifies how many to return – `best_of` must be greater than `n`.

Note: Because this parameter generates many completions, it can quickly consume your token quota. Use carefully and ensure that you have reasonable settings for `max_tokens` and `stop`.

echo

boolean

Indicate whether to echo back the prompt in addition to the completion

frequency_penalty

number

A number between `-2.0` and `2.0`. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

logit_bias

CompletionsLogitBias

Used to modify the likelihood of specified tokens appearing in the completion. Accepts a JSON object that maps tokens (specified by their token ID in the GPT tokenizer) to an associated bias value from -100 to 100. You can use this tokenizer tool to convert text to token IDs. Mathematically, the bias is added to the logits generated by the model prior to sampling.

The exact effect will vary per model, but:

• values between `-1` and `1` should decrease or increase likelihood of selection and
• values like `-100` or `100` should result in a ban or exclusive selection of the relevant token.

As an example, you can pass `{"50256": -100}` to prevent the `<|endoftext|>` token from being generated.

logprobs

integer

The number of most likely output tokens to include the log probabilities of, as well the chosen tokens. For example, if `logprobs` is `5`, the API will return a list of the 5 most likely tokens. The API will always return the `logprob` of the sampled token, so there may be up to `logprobs+1` elements in the response. The maximum value for `logprobs` is `5`.

max_tokens

integer

The maximum number of tokens that can be generated in the completion. The token count of your prompt plus `max_tokens` cannot exceed the model's context length.

metadata

ChatsMetadata

Contains developer-defined tags and values used for filtering completions.

n

integer

Specifies how many completions to generate for each prompt.

Note: Because this parameter generates many completions, it can quickly consume your token quota. Use carefully and ensure that you have reasonable settings for `max_tokens` and `stop`.

presence_penalty

number

A number between `-2.0` and `2.0`. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

seed

integer

The seed for the model request. If specified, OpenAI's system will make a best effort to sample deterministically, such that repeated requests with the same `seed` and parameters should return the same result.

Note: Determinism is not guaranteed, and you should refer to the `system_fingerprint` response parameter to monitor changes in the backend.

stop

ChatsStop

Specifies up to 4 sequences where the API will stop generating further tokens.

stream

boolean

Indicates whether to stream back partial progress. If set, tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a `data: [DONE]` message.

stream_options

StreamOptions

Options for streaming response. Only set this when you set `stream` to `true`.

include_usage

boolean

If set, an additional chunk will be streamed before the data: [DONE] message. The usage field on this chunk shows the token usage statistics for the entire request, and the choices field will always be an empty array. All other chunks will also include a usage field, but with a null value.

suffix

string

Text that comes after a completion of inserted text. On OpenAI, this parameter is only supported for `gpt-3.5-turbo-instruct`.

temperature

number

Specifies what temperature to use for sample, between `0` and `2`. Higher values like `0.8` will make the output more random, while lower values like `0.2` will make it more focused and deterministic.

Note: OpenAI generally recommends altering this or `top_p` but not both.

top_p

number

An alternative to sampling with `temperature`, called nucleus sampling, where the model considers the results of the tokens with `top_p` probability mass. So `0.1` means only the tokens comprising the top 10% probability mass are considered.

Note: OpenAI generally recommends altering this or `temperature` but not both.

user

string

A unique identifier representing your end-user, which can help Services to monitor and detect abuse.

cache

CompletionsCache

The caching configuration for the request. Cache is only supported for non-streaming requests.

enabled

Required *

boolean

Indicates whether caching is enabled.

filter

object

The filtering criteria for caching.

threshold

number

The threshold for caching the request; required if caching is enabled.

router

ModelRouter

The model routing configuration for a request.

family_model

array

The model families to include into the assessment in the case of smart routing.

max_cost

number

The maximum cost of the model to include into the assessment in the case of smart routing.

models

array

The model names to include into the assessment in the case of smart routing.

optimization

string

The model selection optimization parameter. Options are `"efficacy"` (default) and `"cost"`.

quality_tradeoff

number

How much of quality possible to sacrifice during most optimal model selection, value between 0 and 1. Where, e.g. `0.8` means the quality can be reduced by 20%.

region

string

The region of the model to include into the assessment in the case of smart routing.

Response

201 - Created Successfully

id

string

A unique identifier for the completion.

object

string

The object type, which is always `"text_completion"`.

created

integer

The UNIX timestamp (in seconds) of when the completion was created.

model

string

The ID of the model used for the completion.

choices

array

A list of completion choices the model generated for the input prompt.

system_fingerprint

string

The backend configuration that the model runs with. Can be used in conjunction with the seed request parameter to understand when backend changes have been made that might impact determinism.

usage

Usage

Usage information for a model request.

cached

boolean

Indicates whether the request was cached.

400 -

401 -

500 -

Create Embeddings

POST /ml/gateway/v1/embeddings

Generate embeddings based on the provided input using the provided model.

Request

Request Body

CreateEmbeddingsRequest

Create Embedding Request

model

Required *

string

The ID or alias of the model to use.

input

Required *

EmbeddingsInput

Input text to embed, encoded as a string, array of strings, array of integers, or array of integer arrays. The input must not exceed the max input tokens for the model (8192 tokens for OpenAI's `text-embedding-ada-002`) and cannot be an empty string. Any array must be 2048 dimensions or less. Some models may also impose a limit on total number of tokens summed across inputs.

dimensions

integer

The number of dimensions the resulting output embeddings should have. For OpenAI, only supported in `text-embedding-3` and later models.

encoding_format

string

The format to return the embeddings in. Can be either `"float"` or `"base64"`.

user

string

A unique identifier representing your end-user.

Response

201 - Created Successfully

object

string

The object type, which is always `"list"`.

model

string

The ID of the model used to create the embeddings. The same as the `model` provided in the request.

data

array

A list of embedding objects generated by the model.

usage

Usage

Usage information for a model request.

400 -

401 -

500 -

Create Image Generations

POST /ml/gateway/v1/images/generations

Generates an image from a prompt using the provided model.

Request

Request Body

CreateImageRequest

Image Generation Request

model

Required *

string

The ID or alias of model to use for image generation.

prompt

Required *

string

A text description of the desired image(s). For OpenAI, the maximum length is 32000 characters for `gpt-image-1`, and 1000 and 4000 characters respectively for `dall-e-2` and `dall-e-3`.

background

string

The background parameter allows to set transparency for the background for image generation.

In a request this parameter can be `"auto"` (default), `"transparent"` or `"opaque"`. In the response this value not be `"auto"`. When `"auto"` is used, the model will automatically determine the best background for the image. If `"transparent"`, the `"output_format"` needs to support transparency, so it should be set to either `"png"` (default value) or `"webp"`.

For OpenAI, this parameter is only supported for the `gpt-image-1` model.

moderation

string

Control the content-moderation level for images. For OpenAI, this parameter is only supported by the `gpt-image-1` model.

Must be either `"low"` for less restrictive filtering or `"auto"` (default value).

n

integer

The number of images to generate. Must be between `1` and `10`. For OpenAI's `dall-e-3`, `n` only supports a value of `1`.

output_compression

integer

The compression level (0-100%) for the generated images. This parameter is only supported for OpenAI's `gpt-image-1` model with the `"webp"` or `"jpeg"` output formats, and defaults to `100`.

output_format

string

The output format of the image generation. For OpenAI, this parameter is only supported by the `gpt-image-1` model.

In a request this parameter can be `"auto"`, `"png"`, `"webp"`, or `"jpeg"`. In the response this value will not be `"auto"`.

partial_images

integer

The number of partial images to generate. This parameter is used for streaming responses that return partial images. Value must be between `0` and `3`. When set to `0`, the response will be a single image sent in one streaming event.

Note: The final image may be sent before the full number of partial images are generated if the full image is generated more quickly.

quality

string

The quality of the image generated.

In a request this parameter can be `"auto"`, `"hd"`, `"standard"`, `"low"`, `"medium"`, or `"high"`. In the response this value will not be `"auto"`.

`"auto"` (default value) will automatically select the best quality for the given model. For OpenAI:

• `"high"`, `"medium"` and `"low"` are only supported for the `gpt-image-1` model.
• `"hd"` and `"standard"` are supported for the `dall-e-3` model.
• `"standard"` is the only option for the `dall-e-2` model.

response_format

string

The format in which generated images are returned. Must be one of `"url"` or `"b64_json"`. URLs are only valid for 60 minutes after the image has been generated.

For OpenAI, this parameter is only supported by the `dall-e-2` and `dall-e-3` models. This parameter is not supported for the `gpt-image-1` model which will always return base64-encoded images.

size

string

The size of the generated images.

In a request this parameter must be one of:

• `"1024x1024"`, `"1536x1024"` (landscape), `"1024x1536"` (portrait), or `"auto"` (default value) for OpenAI's `gpt-image-1` model.
• `"256x256"`, `"512x512"`, or `"1024x1024"` for OpenAI's `dall-e-2` model.
• `"1024x1024"`, `"1792x1024"`, or `"1024x1792"` for OpenAI's `dall-e-3` model. In the response this value will not be `"auto"`.

style

string

Style of the generated images. Must be one of:

• `"vivid"` causes the model to lean towards generating hyper-real and dramatic images.
• `"natural"` causes the model to produce more natural, less hyper-real looking images.

For OpenAI, this param is only supported for `dall-e-3`.

user

string

A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse.

Response

201 - Image Created Successfully

created

integer

Created is a UNIX timestamp of the time that the model response was created (in seconds).

data

array

A list of the image objects generated by the model.

background

string

The background parameter allows to set transparency for the background for image generation.

In a request this parameter can be `"auto"` (default), `"transparent"` or `"opaque"`. In the response this value not be `"auto"`. When `"auto"` is used, the model will automatically determine the best background for the image. If `"transparent"`, the `"output_format"` needs to support transparency, so it should be set to either `"png"` (default value) or `"webp"`.

For OpenAI, this parameter is only supported for the `gpt-image-1` model.

output_format

string

The output format of the image generation. For OpenAI, this parameter is only supported by the `gpt-image-1` model.

In a request this parameter can be `"auto"`, `"png"`, `"webp"`, or `"jpeg"`. In the response this value will not be `"auto"`.

quality

string

The quality of the image generated.

In a request this parameter can be `"auto"`, `"hd"`, `"standard"`, `"low"`, `"medium"`, or `"high"`. In the response this value will not be `"auto"`.

`"auto"` (default value) will automatically select the best quality for the given model. For OpenAI:

• `"high"`, `"medium"` and `"low"` are only supported for the `gpt-image-1` model.
• `"hd"` and `"standard"` are supported for the `dall-e-3` model.
• `"standard"` is the only option for the `dall-e-2` model.

size

string

The size of the generated images.

In a request this parameter must be one of:

• `"1024x1024"`, `"1536x1024"` (landscape), `"1024x1536"` (portrait), or `"auto"` (default value) for OpenAI's `gpt-image-1` model.
• `"256x256"`, `"512x512"`, or `"1024x1024"` for OpenAI's `dall-e-2` model.
• `"1024x1024"`, `"1792x1024"`, or `"1024x1792"` for OpenAI's `dall-e-3` model. In the response this value will not be `"auto"`.

usage

ImageUsage

The token usage information for the image generation.

For OpenAI, supported by the `gpt-image-1` model only.

400 -

401 -

500 -

List All Models

GET /ml/gateway/v1/models

Lists all configured model details aggregated across all configured providers.

Request

Response

200 - List of all configured models.

object

string

The object type, which is always "list".

data

array

The array of models.

400 -

401 -

500 -

Get Existing Model

GET /ml/gateway/v1/models/{model_uuid}

Retrieves a specific model configuration by model UUID.

Request

Response

200 - Model configuration details.

uuid

string

The unique identifier for the model.

object

string

The object type, which is always "model".

created

integer

The Unix timestamp (in seconds) when the model configuration was created.

owned_by

string

The provider that owns the model.

Format: `"<provider_type>:<provider_name>"

id

string

The official provider-specific server-side unique identifier of the model instance.

alias

string

An optional friendly name for the model. If not set, the default is the value of ID. The Alias can be used by clients to refer to that model in a more convenient or custom manner. When a client provides the alias (e.g., `"my-gpt"`) instead of the official name, the middleware will map the alias back to the underlying ID (e.g., `"gpt-4.1-2025-04-14"`) and execute requests against the correct model.

description

string

A custom user-defined description for the model.

metadata

ModelMetadata

Contains additional configuration for the model.

400 -

401 -

404 -

500 -

Delete Existing Model

DELETE /ml/gateway/v1/models/{model_uuid}

Removes a specific model configuration from the tenant by UUID.

Request

Response

204 - No Content

400 -

401 -

404 -

500 -

Get Current Tenant

GET /ml/gateway/v1/tenant

Retrieves details of the currently authenticated tenant.

Request

Response

200 - Tenant Details

uuid

string

The unique identifier of the tenant.

name

string

The custom name of the tenant.

401 -

404 -

500 -

Replace Current Tenant

PUT /ml/gateway/v1/tenant

Replaces details of the currently authenticated tenant's information.

Request

Request Body

ReplaceTenantRequest

Replacement tenant information details.

name

Required *

string

The new name for the tenant. Currently, the tenant's name is the only mutable field.

Response

200 - Replaced tenant information.

uuid

string

The unique identifier of the tenant.

name

string

The custom name of the tenant.

400 -

401 -

500 -

Update Current Tenant

PATCH /ml/gateway/v1/tenant

Updates details of the currently authenticated tenant's information.

Request

Request Body

UpdateTenantRequest

Updated tenant information details.

name

string

A new name for the tenant. Currently, the tenant's name is the only mutable field.

Response

200 - Updated tenant information.

uuid

string

The unique identifier of the tenant.

name

string

The custom name of the tenant.

400 -

401 -

500 -

Delete Current Tenant

DELETE /ml/gateway/v1/tenant

Deletes an existing tenant.

Request

Response

204 - Tenant Deleted Successfully

401 -

403 -

404 -

500 -

Get Current Tenant Configuration

GET /ml/gateway/v1/tenant/config

Retrieves the current tenant configuration. When `wxai_runtime_instance_id` is set, the response includes `wxai_runtime_plan` and `wxai_runtime_plan_rate_limit` if plan resolution succeeds, or `wxai_runtime_plan_error` if it fails.

Request

Response

200 - OK

default_algorithm

string

The load balancing algorithm to use for distributing requests. Supported algorithms: "least_connections", "weighted_round_robin", "round_robin", "quota_priority"

wxai_runtime_instance_id

string

The watsonx.ai Runtime (WML) service instance CRN or GUID associated with this tenant. When set, the gateway resolves the instance's active pricing plan and applies plan-based rate limits.

wxai_runtime_plan

string

The detected watsonx.ai Runtime pricing/tier plan (for example, "lite", "v2-standard", or "v2-professional"). Present only when wxai_runtime_instance_id is set and plan resolution succeeds.

wxai_runtime_plan_rate_limit

WxAIRuntimePlanRateLimit

The request rate limit applied to the gateway based on the resolved watsonx.ai Runtime plan. Present only when wxai_runtime_instance_id is set and plan resolution succeeds.

wxai_runtime_plan_error

string

Present only if plan resolution failed (for example, if the instance ID is invalid or inaccessible with the provided credentials).

401 -

500 -

Update Current Tenant Configuration

PUT /ml/gateway/v1/tenant/config

Updates stored tenant configuration fields. When `wxai_runtime_instance_id` is set, the gateway queries the watsonx.ai machine learning API to resolve the instance's active tier, plan name, and rate limits. The response includes `wxai_runtime_plan` and `wxai_runtime_plan_rate_limit` if plan resolution succeeds, or `wxai_runtime_plan_error` if it fails. Set `wxai_runtime_instance_id` to an empty string to unlink the current instance.

Request

Request Body

TenantConfigRequest

Tenant configuration fields to update

default_algorithm

string

The load balancing algorithm to use for distributing requests. Supported algorithms: "least_connections", "weighted_round_robin", "round_robin", "quota_priority"

wxai_runtime_instance_id

string

The watsonx.ai Runtime (WML) service instance CRN or GUID to associate with this tenant. Set to an empty string to unlink the current instance.

Response

200 - OK

default_algorithm

string

The load balancing algorithm to use for distributing requests. Supported algorithms: "least_connections", "weighted_round_robin", "round_robin", "quota_priority"

wxai_runtime_instance_id

string

The watsonx.ai Runtime (WML) service instance CRN or GUID associated with this tenant. When set, the gateway resolves the instance's active pricing plan and applies plan-based rate limits.

wxai_runtime_plan

string

The detected watsonx.ai Runtime pricing/tier plan (for example, "lite", "v2-standard", or "v2-professional"). Present only when wxai_runtime_instance_id is set and plan resolution succeeds.

wxai_runtime_plan_rate_limit

WxAIRuntimePlanRateLimit

The request rate limit applied to the gateway based on the resolved watsonx.ai Runtime plan. Present only when wxai_runtime_instance_id is set and plan resolution succeeds.

wxai_runtime_plan_error

string

Present only if plan resolution failed (for example, if the instance ID is invalid or inaccessible with the provided credentials).

400 -

401 -

500 -

Delete Current Tenant Configuration

DELETE /ml/gateway/v1/tenant/config

Resets all tenant configuration fields, including `default_algorithm` and `wxai_runtime_instance_id`. Returns `204 No Content` on success.

Request

Response

204 - Configuration deleted successfully

401 -

500 -

Create Tenant JWT

POST /ml/gateway/v1/tenant/jwt

Creates a JWT token for direct access to backend services using the tenant UUID.

Request

Response

200 - JWT Token

token

string

The raw authorization token string.

400 -

401 -

500 -

List All Policies

GET /ml/gateway/v1/policies

Lists all the existing policies for the tenant.

Request

Response

200 - List of Policies

object

string

The object type, always `"list"`.

data

array

The array of tenant policies.

400 -

401 -

500 -

Create New Policy

POST /ml/gateway/v1/policies

Creates a new policy.

Request

Request Body

CreateTenantPolicyRequest

Policy configuration

subject

Required *

string

A unique identifier for the party subject to this resource access policy. For IBM Cloud IAM this can be the Access Group ID.

resource

Required *

string

A unique identifier for the resource this policy is restricting access to.

Format: `"<resource_type>:<resource_uuid>"` Currently, the supported resource types are: `tenant`, `provider`, `model`, and `policy`.

action

Required *

string

The action to perform on the policy. Currently only one of `"read"` or `"write"` are supported.

effect

string

The effect that the policy is to have, either `"allow"` or `"deny"`.

Response

201 - Policy Created Successfully

uuid

string

The unique identifier for the policy.

subject

string

A unique identifier for the party subject to this resource access policy. For IBM Cloud IAM this can be the Access Group ID.

resource

string

A unique identifier for the resource this policy is restricting access to.

Format: `"<resource_type>:<resource_uuid>"` Currently, the supported resource types are: `tenant`, `provider`, `model`, and `policy`.

action

string

The action to perform on the policy. Currently only one of `"read"` or `"write"` are supported.

effect

string

The effect that the policy is to have, either `"allow"` or `"deny"`.

400 -

401 -

500 -

Get Specific Policy

GET /ml/gateway/v1/policies/{policy_uuid}

Retrieves a specific policy by its UUID.

Request

Response

200 - Policy Retrieved Successfully

uuid

string

The unique identifier for the policy.

subject

string

A unique identifier for the party subject to this resource access policy. For IBM Cloud IAM this can be the Access Group ID.

resource

string

A unique identifier for the resource this policy is restricting access to.

Format: `"<resource_type>:<resource_uuid>"` Currently, the supported resource types are: `tenant`, `provider`, `model`, and `policy`.

action

string

The action to perform on the policy. Currently only one of `"read"` or `"write"` are supported.

effect

string

The effect that the policy is to have, either `"allow"` or `"deny"`.

400 -

401 -

404 -

500 -

Update Existing Policy

PUT /ml/gateway/v1/policies/{policy_uuid}

Updates an existing policy with the specified UUID. Only provided fields are updated; omitted fields retain their current values.

Request

Request Body

UpdateTenantPolicyRequest

Updated Policy Configuration

subject

string

A unique identifier for the party subject to this resource access policy. For IBM Cloud IAM this can be the Access Group ID.

resource

string

A unique identifier for the resource this policy is restricting access to.

Format: `"<resource_type>:<resource_uuid>"` Currently, the supported resource types are: `tenant`, `provider`, `model`, and `policy`.

action

string

The action to perform on the policy. Currently only one of `"read"` or `"write"` are supported.

effect

string

The effect that the policy is to have, either `"allow"` or `"deny"`.

Response

200 - Policy Updated Successfully

uuid

string

The unique identifier for the policy.

subject

string

A unique identifier for the party subject to this resource access policy. For IBM Cloud IAM this can be the Access Group ID.

resource

string

A unique identifier for the resource this policy is restricting access to.

Format: `"<resource_type>:<resource_uuid>"` Currently, the supported resource types are: `tenant`, `provider`, `model`, and `policy`.

action

string

The action to perform on the policy. Currently only one of `"read"` or `"write"` are supported.

effect

string

The effect that the policy is to have, either `"allow"` or `"deny"`.

400 -

401 -

404 -

500 -

Delete Existing Policy

DELETE /ml/gateway/v1/policies/{policy_uuid}

Deletes an existing policy with the specified UUID from the tenant.

Request

Response

204 - No Content

400 -

401 -

500 -

List All Providers

GET /ml/gateway/v1/providers

Lists all configured model providers for the tenant.

Request

Response

200 - List of model providers

object

string

The object type, always "list".

data

array

The array of model providers.

400 -

401 -

500 -

Create New Anthropic Provider

POST /ml/gateway/v1/providers/anthropic

Creates a new Anthropic model provider configuration with the supplied details.

Request

Request Body

AnthropicProviderRequest

Anthropic provider configuration details.

name

Required *

string

A custom name for the Anthropic model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the Anthropic model provider instance.

data

AnthropicConfig

Configuration details for an Anthropic provider.

apikey

Required *

string

The required authentication key for accessing the Anthropic API.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

201 - Created Anthropic provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Create New Azure OpenAI Provider

POST /ml/gateway/v1/providers/azure_openai

Creates a new Azure OpenAI model provider configuration with the supplied details.

Request

Request Body

AzureOpenAIProviderRequest

Azure OpenAI provider configuration details.

name

Required *

string

A custom name for the Azure OpenAI model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the Azure OpenAI model provider instance.

data

AzureOpenAIConfig

Configuration details for an Azure OpenAI provider.

account_name

string

The Azure account name; required to use `/v1/providers/{provider_uuid}/models`.

api_version

Required *

string

Version of the Azure OpenAI API to use (default: `"2024-10-21"`).

apikey

Required *

string

The required authentication key for accessing Azure OpenAI services.

resource_group_name

string

The Azure resource group name; required to use `/v1/providers/{provider_uuid}/models`.

resource_name

Required *

string

The Azure OpenAI resource to connect to.

subscription_id

string

The Azure subscription ID; required to use `/v1/providers/{provider_uuid}/models`.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

201 - Created Azure OpenAI provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Create New AWS Bedrock Provider

POST /ml/gateway/v1/providers/bedrock

Creates a new AWS Bedrock model provider configuration with the supplied details.

Request

Request Body

AWSBedrockProviderRequest

AWS Bedrock provider configuration details.

name

Required *

string

A custom name for the AWS Bedrock model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the AWS Bedrock model provider instance.

data

AWSBedrockConfig

Configuration details for an AWS Bedrock provider.

access_key_id

Required *

string

The AWS access key ID required to authenticate with the Bedrock API.

base_url

string

Overrides the default AWS Bedrock Runtime API endpoint with the provided URL.

region

Required *

string

The AWS region where the Bedrock API is hosted.

secret_access_key

Required *

string

The AWS secret access key required to authenticate with the Bedrock API.

session_token

string

Optional AWS session token for temporary credentials.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

201 - Created AWS Bedrock provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Create New Cerebras Provider

POST /ml/gateway/v1/providers/cerebras

Creates a new Cerebras model provider configuration with the supplied details.

Request

Request Body

CerebrasProviderRequest

Cerebras provider configuration details.

name

Required *

string

A custom name for the Cerebras model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the Cerebras model provider instance.

data

CerebrasConfig

Configuration details for a Cerebras provider.

apikey

Required *

string

The required authentication key for accessing the Cerebras API.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

201 - Created Cerebras provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Create New Cohere Provider

POST /ml/gateway/v1/providers/cohere

Creates a new Cohere model provider configuration with the supplied details.

Request

Request Body

CohereProviderRequest

Cohere provider configuration details.

description

string

A custom user-defined description for the model provider.

name

Required *

string

Name can only contain alphanumeric characters, single spaces (no consecutive spaces), underscores (_), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

data

CohereConfig

Configuration details for a Cohere provider.

apikey

Required *

string

The required authentication key for accessing the Cohere API.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

201 - Created Cohere provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Create New Gemini Provider

POST /ml/gateway/v1/providers/gemini

Creates a new Google Gemini model provider configuration with the supplied details.

Request

Request Body

GeminiProviderRequest

Gemini provider configuration details.

name

Required *

string

A custom name for the Gemini model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the Gemini model provider instance.

data

GeminiConfig

Configuration details for a Gemini provider.

apikey

Required *

string

The required Google AI Studio API key for accessing the Gemini API.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

201 - Created Gemini provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Create New Groq Provider

POST /ml/gateway/v1/providers/groq

Creates a new Groq model provider configuration with the supplied details.

Request

Request Body

GroqProviderRequest

Groq Provider Configuration Details.

name

Required *

string

A custom name for the Groq model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the Groq model provider instance.

data

GroqConfig

Contains the credential details for configuring the provider instance.

apikey

Required *

string

APIKey is the required authentication key for accessing the Groq API.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

201 - Created Groq Provider Configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Create New Mistral Provider

POST /ml/gateway/v1/providers/mistral

Creates a new Mistral model provider configuration with the supplied details.

Request

Request Body

MistralProviderRequest

Mistral provider configuration details.

description

string

A custom user-defined description for the model provider.

name

Required *

string

Name can only contain alphanumeric characters, single spaces (no consecutive spaces), underscores (_), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

data

MistralConfig

Configuration details for a Mistral provider.

apikey

Required *

string

The required authentication key for accessing the Mistral API.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

201 - Created Mistral provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Create New Nvidia NIM Provider

POST /ml/gateway/v1/providers/nim

Creates a new Nvidia NIM model provider configuration with the supplied details.

Request

Request Body

NvidiaNIMProviderRequest

Nvidia NIM provider configuration details.

name

Required *

string

A custom name for the Nvidia NIM model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the Nvidia NIM model provider instance.

data

NvidiaNIMConfig

Configuration details for an Nvidia NIM provider.

apikey

Required *

string

The required authentication key for accessing the Nvidia NIM API.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

201 - Created Nvidia NIM provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Create New OpenAI Provider

POST /ml/gateway/v1/providers/openai

Creates a new OpenAI model provider configuration with the supplied details.

Request

Request Body

OpenAIProviderRequest

OpenAI provider configuration details.

name

Required *

string

A custom name for the OpenAI model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the OpenAI model provider instance.

data

OpenAIConfig

Configuration details for an OpenAI provider.

apikey

Required *

string

The required authentication key for accessing the OpenAI API.

base_url

string

Override the URL used to access the OpenAI provider services. This URL can point to any OpenAI-compatible model provider service.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

201 - Created OpenAI provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Create New Red Hat AI Provider

POST /ml/gateway/v1/providers/redhatai

Creates a new Red Hat AI model provider configuration with the supplied details.

Request

Request Body

RedHatAIProviderRequest

Red Hat AI provider configuration details.

name

Required *

string

A custom name for the Red Hat AI model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the Red Hat AI model provider instance.

data

RedHatAIConfig

Configuration details for a Red Hat AI provider.

apikey

string

The authentication key for accessing the Red Hat AI API. Required when using the IBM-hosted endpoint (`us-east.rhai.ibm.com`). May be omitted for standalone RHOSAI or MaaS deployments.

base_url

string

Overrides the default Red Hat AI API endpoint with the provided URL. Defaults to `https://us-east.rhai.ibm.com` when not specified.

project_id

string

The project identifier for the Red Hat AI service. Required when using the IBM-hosted endpoint (`us-east.rhai.ibm.com`). May be omitted for standalone RHOSAI or MaaS deployments.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

201 - Created Red Hat AI provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Create New IBM Watsonx.ai Provider

POST /ml/gateway/v1/providers/watsonxai

Creates a new IBM Watsonx.ai model provider configuration with the supplied details.

Request

Request Body

WatsonxAIProviderRequest

IBM Watsonx.ai provider configuration details.

name

Required *

string

A custom name for the Watsonx.ai model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the Watsonx.ai model provider instance.

data

WatsonxAIConfig

Configuration details for a WatsonX.ai provider.

api_version

string

Overrides the WatsonX.ai API version to use.

apikey

string

Optional authentication key for accessing WatsonX.ai services.

auth_url

string

Overrides the URL to use for IBM Cloud IAM authentication.

base_url

Required *

string

Overrides the URL to use to access the IBM WatsonX.ai services.

project_id

string

The IBM WatsonX.ai project ID (required if `space_id` is not provided).

space_id

string

The IBM WatsonX.ai space ID (required if `project_id` is not provided).

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

201 - Created IBM Watsonx.ai provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Create New xAI Provider

POST /ml/gateway/v1/providers/xai

Creates a new xAI model provider configuration with the supplied details.

Request

Request Body

XaiProviderRequest

xAI provider configuration details.

description

string

A custom user-defined description for the model provider.

name

Required *

string

Name can only contain alphanumeric characters, single spaces (no consecutive spaces), underscores (_), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

data

XaiConfig

Configuration details for an xAI provider.

apikey

Required *

string

The required authentication key for accessing the xAI API.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

201 - Created xAI provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Find Existing Providers

GET /ml/gateway/v1/providers/search

Searches for providers by name.

Request

Path Parameters

Query Parameters

name

string

Provider name to search for

Response

200 - List of matching Providers

object

string

The object type, always "list".

data

array

The array of model providers.

400 -

401 -

500 -

Validate Provider Credentials

POST /ml/gateway/v1/providers/validate

Validates provider credentials before creating a provider instance. Returns 200 OK with valid field indicating success or failure. Supports both direct credentials via the data field and credential references via data_reference. If both are provided, data takes precedence.

Request

Request Body

ProvidersValidateRequest

Validation Request

type

Required *

string

The type of provider to validate (e.g., "openai", "anthropic", "watsonxai").

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

data

object

The credential details for validating the provider. This should be a JSON object matching the provider's configuration schema.

Response

200 - Validation result (check valid field for success/failure)

provider_type

string

The provider type that was validated.

valid

boolean

Indicates whether the credential validation was successful. true = credentials are valid, false = validation failed

message

string

A human-readable message describing the validation result. For successful validations (valid=true), this confirms the credentials are valid. For failed validations (valid=false), this provides details about why validation failed.

400 -

401 -

403 -

500 -

Get Existing Provider

GET /ml/gateway/v1/providers/{provider_uuid}

Retrieves the details of an existing model provider with the specified UUID.

Request

Response

200 - Provider Details

uuid

string

The unique identifier of the model provider.

name

string

The custom name of the model provider.

type

string

The provider type that the credential is for.

description

string

The custom description of the model provider.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

data

object

Data to be mapped to a provider credential config.

models

array

The configured models for this provider.

400 -

401 -

404 -

500 -

Delete Existing Provider

DELETE /ml/gateway/v1/providers/{provider_uuid}

Deletes an existing model provider configuration with the specified UUID.

Request

Response

204 - No Content

400 -

401 -

404 -

500 -

Replace Existing Anthropic Provider

PUT /ml/gateway/v1/providers/{provider_uuid}/anthropic

Replace information for an existing Anthropic provider configuration with new details.

Request

Request Body

AnthropicProviderRequest

Replacement Anthropic model provider configuration details.

name

Required *

string

A custom name for the Anthropic model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the Anthropic model provider instance.

data

AnthropicConfig

Configuration details for an Anthropic provider.

apikey

Required *

string

The required authentication key for accessing the Anthropic API.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

200 - Replaced Anthropic provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Replace Existing Azure OpenAI Provider

PUT /ml/gateway/v1/providers/{provider_uuid}/azure_openai

Replace information for an existing Azure OpenAI provider configuration with new details.

Request

Request Body

AzureOpenAIProviderRequest

Replacement Azure OpenAI model provider configuration details.

name

Required *

string

A custom name for the Azure OpenAI model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the Azure OpenAI model provider instance.

data

AzureOpenAIConfig

Configuration details for an Azure OpenAI provider.

account_name

string

The Azure account name; required to use `/v1/providers/{provider_uuid}/models`.

api_version

Required *

string

Version of the Azure OpenAI API to use (default: `"2024-10-21"`).

apikey

Required *

string

The required authentication key for accessing Azure OpenAI services.

resource_group_name

string

The Azure resource group name; required to use `/v1/providers/{provider_uuid}/models`.

resource_name

Required *

string

The Azure OpenAI resource to connect to.

subscription_id

string

The Azure subscription ID; required to use `/v1/providers/{provider_uuid}/models`.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

200 - Replaced Azure OpenAI provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Replace Existing AWS Bedrock Provider

PUT /ml/gateway/v1/providers/{provider_uuid}/bedrock

Replaces information for an existing AWS Bedrock provider configuration with new details.

Request

Request Body

AWSBedrockProviderRequest

Replacement AWS Bedrock model provider configuration details.

name

Required *

string

A custom name for the AWS Bedrock model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the AWS Bedrock model provider instance.

data

AWSBedrockConfig

Configuration details for an AWS Bedrock provider.

access_key_id

Required *

string

The AWS access key ID required to authenticate with the Bedrock API.

base_url

string

Overrides the default AWS Bedrock Runtime API endpoint with the provided URL.

region

Required *

string

The AWS region where the Bedrock API is hosted.

secret_access_key

Required *

string

The AWS secret access key required to authenticate with the Bedrock API.

session_token

string

Optional AWS session token for temporary credentials.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

200 - Replaced AWS Bedrock provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Replace Existing Cerebras Provider

PUT /ml/gateway/v1/providers/{provider_uuid}/cerebras

Replaces information for an existing Cerebras model provider configuration with new details.

Request

Request Body

CerebrasProviderRequest

Replacement Cerebras provider configuration details.

name

Required *

string

A custom name for the Cerebras model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the Cerebras model provider instance.

data

CerebrasConfig

Configuration details for a Cerebras provider.

apikey

Required *

string

The required authentication key for accessing the Cerebras API.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

200 - Replaced Cerebras provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Replace Existing Cohere Provider

PUT /ml/gateway/v1/providers/{provider_uuid}/cohere

Replaces information for an existing Cohere provider configuration with new details.

Request

Request Body

CohereProviderRequest

Replacement Cohere provider configuration details.

description

string

A custom user-defined description for the model provider.

name

Required *

string

Name can only contain alphanumeric characters, single spaces (no consecutive spaces), underscores (_), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

data

CohereConfig

Configuration details for a Cohere provider.

apikey

Required *

string

The required authentication key for accessing the Cohere API.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

200 - Replaced Cohere provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Replace Existing Gemini Provider

PUT /ml/gateway/v1/providers/{provider_uuid}/gemini

Replace information for an existing Google Gemini provider configuration with new details.

Request

Request Body

GeminiProviderRequest

Replacement Gemini model provider configuration details.

name

Required *

string

A custom name for the Gemini model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the Gemini model provider instance.

data

GeminiConfig

Configuration details for a Gemini provider.

apikey

Required *

string

The required Google AI Studio API key for accessing the Gemini API.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

200 - Replaced Gemini provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Replace Existing Groq Provider

PUT /ml/gateway/v1/providers/{provider_uuid}/groq

Replace information for an existing Groq provider configuration with new details.

Request

Request Body

GroqProviderRequest

Replacement Groq model provider configuration details.

name

Required *

string

A custom name for the Groq model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the Groq model provider instance.

data

GroqConfig

Contains the credential details for configuring the provider instance.

apikey

Required *

string

APIKey is the required authentication key for accessing the Groq API.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

200 - Replaced Groq provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Replace Existing Mistral Provider

PUT /ml/gateway/v1/providers/{provider_uuid}/mistral

Replaces information for an existing Mistral provider configuration with new details.

Request

Request Body

MistralProviderRequest

Replacement Mistral provider configuration details.

description

string

A custom user-defined description for the model provider.

name

Required *

string

Name can only contain alphanumeric characters, single spaces (no consecutive spaces), underscores (_), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

data

MistralConfig

Configuration details for a Mistral provider.

apikey

Required *

string

The required authentication key for accessing the Mistral API.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

200 - Replaced Mistral provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Replace Existing Nvidia NIM Provider

PUT /ml/gateway/v1/providers/{provider_uuid}/nim

Replaces information for an existing Nvidia NIM provider configuration with new details.

Request

Request Body

NvidiaNIMProviderRequest

Replacement Nvidia NIM provider configuration details.

name

Required *

string

A custom name for the Nvidia NIM model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the Nvidia NIM model provider instance.

data

NvidiaNIMConfig

Configuration details for an Nvidia NIM provider.

apikey

Required *

string

The required authentication key for accessing the Nvidia NIM API.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

200 - Replaced Nvidia NIM provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Replace Existing OpenAI Provider

PUT /ml/gateway/v1/providers/{provider_uuid}/openai

Replaces information for an existing OpenAI provider configuration with new details.

Request

Request Body

OpenAIProviderRequest

Replacement OpenAI provider configuration details.

name

Required *

string

A custom name for the OpenAI model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the OpenAI model provider instance.

data

OpenAIConfig

Configuration details for an OpenAI provider.

apikey

Required *

string

The required authentication key for accessing the OpenAI API.

base_url

string

Override the URL used to access the OpenAI provider services. This URL can point to any OpenAI-compatible model provider service.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

200 - Replaced OpenAI provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Replace Existing Red Hat AI Provider

PUT /ml/gateway/v1/providers/{provider_uuid}/redhatai

Replace information for an existing Red Hat AI provider configuration with new details.

Request

Request Body

RedHatAIProviderRequest

Replacement Red Hat AI model provider configuration details.

name

Required *

string

A custom name for the Red Hat AI model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the Red Hat AI model provider instance.

data

RedHatAIConfig

Configuration details for a Red Hat AI provider.

apikey

string

The authentication key for accessing the Red Hat AI API. Required when using the IBM-hosted endpoint (`us-east.rhai.ibm.com`). May be omitted for standalone RHOSAI or MaaS deployments.

base_url

string

Overrides the default Red Hat AI API endpoint with the provided URL. Defaults to `https://us-east.rhai.ibm.com` when not specified.

project_id

string

The project identifier for the Red Hat AI service. Required when using the IBM-hosted endpoint (`us-east.rhai.ibm.com`). May be omitted for standalone RHOSAI or MaaS deployments.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

200 - Replaced Red Hat AI provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Replace Existing IBM Watsonx.ai Provider

PUT /ml/gateway/v1/providers/{provider_uuid}/watsonxai

Replaces information for an existing IBM WatsonX.ai provider configuration with new details.

Request

Request Body

WatsonxAIProviderRequest

Replacement IBM Watsonx.ai provider configuration details.

name

Required *

string

A custom name for the Watsonx.ai model provider instance. The name can only contain alphanumeric characters, single spaces (no consecutive spaces), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

description

string

A custom description for the Watsonx.ai model provider instance.

data

WatsonxAIConfig

Configuration details for a WatsonX.ai provider.

api_version

string

Overrides the WatsonX.ai API version to use.

apikey

string

Optional authentication key for accessing WatsonX.ai services.

auth_url

string

Overrides the URL to use for IBM Cloud IAM authentication.

base_url

Required *

string

Overrides the URL to use to access the IBM WatsonX.ai services.

project_id

string

The IBM WatsonX.ai project ID (required if `space_id` is not provided).

space_id

string

The IBM WatsonX.ai space ID (required if `project_id` is not provided).

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

200 - Replaced IBM WatsonX.ai provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

Replace Existing xAI Provider

PUT /ml/gateway/v1/providers/{provider_uuid}/xai

Replaces information for an existing xAI provider configuration with new details.

Request

Request Body

XaiProviderRequest

Replacement xAI provider configuration details.

description

string

A custom user-defined description for the model provider.

name

Required *

string

Name can only contain alphanumeric characters, single spaces (no consecutive spaces), underscores (_), hyphens (-), parentheses (), and square brackets []. No leading or trailing spaces are allowed.

data

XaiConfig

Configuration details for an xAI provider.

apikey

Required *

string

The required authentication key for accessing the xAI API.

data_reference

DataReference

Data Reference is a reference to a remote credential store. For example, an IBM Cloud Secrets Manager secret. The value in the remote store is expected to be a JSON representation of the `data` field of a provider request.

resource

string

A unique identifier for the remote store that references the credential data object.

Response

200 - Replaced xAI provider configuration.

uuid

string

The unique identifier of the model provider.

name

string

The name of the model provider.

type

string

The model provider type of this instance.

description

string

A custom description of the provider.

400 -

401 -

500 -

List Provider Models

GET /ml/gateway/v1/providers/{provider_uuid}/models

Lists all model configurations for the specified provider.

Request

Response

200 - List of Models

object

string

The object type, which is always "list".

data

array

The array of models.

400 -

401 -

500 -

Create New Model

POST /ml/gateway/v1/providers/{provider_uuid}/models

Creates a new model configuration for the specified provider.

Request

Request Body

CreateModelRequest

Model Configuration

id

Required *

string

The unique identifier of a model, as specified by the model provider that its hosting and providing it. For example, ID could be `"gpt-4.1"` or a specific snapshot like `"gpt-4.1-2025-04-14"`, which are official server-side IDs for a model offered by the OpenAI provider.

alias

string

An optional friendly name for the model. If not set, the default is the value of ID. The Alias can be used by clients to refer to that model in a more convenient or custom manner. When a client provides the alias (e.g., `"my-gpt"`) instead of the official name, the middleware will map the alias back to the underlying ID (e.g., `"gpt-4.1-2025-04-14"`) and execute requests against the correct model.

description

string

A custom user-defined description for the model.

metadata

ModelMetadata

Contains additional configuration for the model.

cost

number

The cost per 1000 tokens for the model. This represents the pricing in USD for processing 1000 tokens. A nil value indicates the cost is not available or applicable.

model_family

string

The family or series that this model belongs to. Examples:

• "gpt-3.5" for GPT-3.5 series models
• "gpt-4" for GPT-4 series models
• "llama-2" for Llama 2 series models

recommender_label

string

When calling the Recommender API, this value is used to map to the supported model in the Recommender API. For example, "openai-gpt-4o-mini" could be the recommender label for a specific model.

region

string

The region where this model is deployed in. This indicates the geographical location of the model deployment. Important for data residency requirements and latency considerations.

batch

boolean

Marks the model as eligible or preferred for batch requests.

context_window

integer

The maximum number of tokens the model can process in a single request.

Response

201 - Model Created

uuid

string

The unique identifier for the model.

object

string

The object type, which is always "model".

created

integer

The Unix timestamp (in seconds) when the model configuration was created.

owned_by

string

The provider that owns the model.

Format: `"<provider_type>:<provider_name>"

id

string

The official provider-specific server-side unique identifier of the model instance.

alias

string

An optional friendly name for the model. If not set, the default is the value of ID. The Alias can be used by clients to refer to that model in a more convenient or custom manner. When a client provides the alias (e.g., `"my-gpt"`) instead of the official name, the middleware will map the alias back to the underlying ID (e.g., `"gpt-4.1-2025-04-14"`) and execute requests against the correct model.

description

string

A custom user-defined description for the model.

metadata

ModelMetadata

Contains additional configuration for the model.

400 -

401 -

500 -

Replace Existing Model

PUT /ml/gateway/v1/providers/{provider_uuid}/models/{model_uuid}

Replace an existing model configuration with the specified UUID with the supplied information.

Request

Request Body

ReplaceModelRequest

Replacement model configuration

id

Required *

string

The unique identifier of a model, as specified by the model provider that its hosting and providing it. For example, ID could be `"gpt-4.1"` or a specific snapshot like `"gpt-4.1-2025-04-14"`, which are official server-side IDs for a model offered by the OpenAI provider.

alias

string

An optional friendly name for the model. If not set, the default is the value of ID. The Alias can be used by clients to refer to that model in a more convenient or custom manner. When a client provides the alias (e.g., `"my-gpt"`) instead of the official name, the middleware will map the alias back to the underlying ID (e.g., `"gpt-4.1-2025-04-14"`) and execute requests against the correct model.

description

string

A custom user-defined description for the model.

metadata

ModelMetadata

Contains additional configuration for the model.

cost

number

The cost per 1000 tokens for the model. This represents the pricing in USD for processing 1000 tokens. A nil value indicates the cost is not available or applicable.

model_family

string

The family or series that this model belongs to. Examples:

• "gpt-3.5" for GPT-3.5 series models
• "gpt-4" for GPT-4 series models
• "llama-2" for Llama 2 series models

recommender_label

string

When calling the Recommender API, this value is used to map to the supported model in the Recommender API. For example, "openai-gpt-4o-mini" could be the recommender label for a specific model.

region

string

The region where this model is deployed in. This indicates the geographical location of the model deployment. Important for data residency requirements and latency considerations.

batch

boolean

Marks the model as eligible or preferred for batch requests.

context_window

integer

The maximum number of tokens the model can process in a single request.

Response

201 - Replaced model configuration

uuid

string

The unique identifier for the model.

object

string

The object type, which is always "model".

created

integer

The Unix timestamp (in seconds) when the model configuration was created.

owned_by

string

The provider that owns the model.

Format: `"<provider_type>:<provider_name>"

id

string

The official provider-specific server-side unique identifier of the model instance.

alias

string

An optional friendly name for the model. If not set, the default is the value of ID. The Alias can be used by clients to refer to that model in a more convenient or custom manner. When a client provides the alias (e.g., `"my-gpt"`) instead of the official name, the middleware will map the alias back to the underlying ID (e.g., `"gpt-4.1-2025-04-14"`) and execute requests against the correct model.

description

string

A custom user-defined description for the model.

metadata

ModelMetadata

Contains additional configuration for the model.

400 -

401 -

404 -

500 -

Update Existing Model

PATCH /ml/gateway/v1/providers/{provider_uuid}/models/{model_uuid}

Update information for an existing specific model configuration with the specified UUID.

Request

Request Body

UpdateModelRequest

Updated model configuration

id

Required *

string

The unique identifier of a model, as specified by the model provider that its hosting and providing it. For example, ID could be `"gpt-4.1"` or a specific snapshot like `"gpt-4.1-2025-04-14"`, which are official server-side IDs for a model offered by the OpenAI provider.

alias

string

An optional friendly name for the model. If not set, the default is the value of ID. The Alias can be used by clients to refer to that model in a more convenient or custom manner. When a client provides the alias (e.g., `"my-gpt"`) instead of the official name, the middleware will map the alias back to the underlying ID (e.g., `"gpt-4.1-2025-04-14"`) and execute requests against the correct model.

description

string

A custom user-defined description for the model.

metadata

ModelMetadata

Contains additional configuration for the model.

cost

number

The cost per 1000 tokens for the model. This represents the pricing in USD for processing 1000 tokens. A nil value indicates the cost is not available or applicable.

model_family

string

The family or series that this model belongs to. Examples:

• "gpt-3.5" for GPT-3.5 series models
• "gpt-4" for GPT-4 series models
• "llama-2" for Llama 2 series models

recommender_label

string

When calling the Recommender API, this value is used to map to the supported model in the Recommender API. For example, "openai-gpt-4o-mini" could be the recommender label for a specific model.

region

string

The region where this model is deployed in. This indicates the geographical location of the model deployment. Important for data residency requirements and latency considerations.

batch

boolean

Marks the model as eligible or preferred for batch requests.

context_window

integer

The maximum number of tokens the model can process in a single request.

Response

200 - Updated model configuration

uuid

string

The unique identifier for the model.

object

string

The object type, which is always "model".

created

integer

The Unix timestamp (in seconds) when the model configuration was created.

owned_by

string

The provider that owns the model.

Format: `"<provider_type>:<provider_name>"

id

string

The official provider-specific server-side unique identifier of the model instance.

alias

string

An optional friendly name for the model. If not set, the default is the value of ID. The Alias can be used by clients to refer to that model in a more convenient or custom manner. When a client provides the alias (e.g., `"my-gpt"`) instead of the official name, the middleware will map the alias back to the underlying ID (e.g., `"gpt-4.1-2025-04-14"`) and execute requests against the correct model.

description

string

A custom user-defined description for the model.

metadata

ModelMetadata

Contains additional configuration for the model.

400 -

401 -

404 -

500 -

Delete Existing Model

DELETE /ml/gateway/v1/providers/{provider_uuid}/models/{model_uuid}

Deletes an existing model configuration with the specified UUID.

Request

Response

204 - No Content

400 -

401 -

500 -

List Available Models

GET /ml/gateway/v1/providers/{provider_uuid}/models_available

Lists all models available for the specified provider.

Request

Response

200 - List of Information on Available Models

object

string

The object type, always "list".

data

array

The array of model responses.

400 -

401 -

500 -

List Load Balancers

GET /ml/gateway/v1/load_balancers

Lists all load balancers (both user-defined and system-generated) for the tenant.

Request

Response

200 - List of Load Balancers

object

string

The object type, which is always "list".

data

array

The list of load balancers.

401 -

500 -

Create Load Balancer

POST /ml/gateway/v1/load_balancers

Creates a new user-defined load balancer for the tenant.

Request

Request Body

LoadBalancersLoadBalancerPrototype

Load Balancer Creation Request

name

Required *

string

A custom name for the new load balancer.

alias

Required *

string

The alias that clients will use to reference this load balancer. This alias must be unique within the tenant and cannot conflict with existing model aliases.

config

LoadBalancersLoadBalancerConfig

Optional algorithm-specific configuration parameters. The structure of this object depends on the selected algorithm.

default_weight

integer

DefaultWeight is the default weight assigned to backends when using weighted algorithms

Used by: weighted_round_robin

algorithm

Required *

string

The load balancing algorithm to use for distributing requests. Supported algorithms: "least_connections", "weighted_round_robin", "round_robin", "quota_priority"

backends

array

Optional list of backends to create along with the load balancer. If provided, backends will be created in the same transaction as the load balancer.

Response

201 - Created Load Balancer

uuid

string

The unique identifier for the load balancer.

name

string

The name of the load balancer.

alias

string

The alias that clients use to reference this load balancer.

algorithm

string

The load balancing algorithm used for distributing requests.

config

object

Algorithm-specific configuration parameters as raw JSON.

is_user_defined

boolean

Indicates whether this load balancer was created by a user (true) or generated from model aliases (false).

backends

array

The list of backends associated with this load balancer. This field is populated when requesting a specific load balancer or when explicitly requested.

stats

object

Optional usage and rate-limit statistics for this load balancer over the last 24 hours.

created_at

string

The Unix timestamp (in seconds) when the load balancer was created.

updated_at

string

The Unix timestamp (in seconds) when the load balancer was last updated.

400 -

401 -

500 -

Get Load Balancer

GET /ml/gateway/v1/load_balancers/{load_balancer_uuid}

Retrieves a specific load balancer by ID (supports both user-defined and system-generated load balancers).

Request

Response

200 - Load Balancer

uuid

string

The unique identifier for the load balancer.

name

string

The name of the load balancer.

alias

string

The alias that clients use to reference this load balancer.

algorithm

string

The load balancing algorithm used for distributing requests.

config

object

Algorithm-specific configuration parameters as raw JSON.

is_user_defined

boolean

Indicates whether this load balancer was created by a user (true) or generated from model aliases (false).

backends

array

The list of backends associated with this load balancer. This field is populated when requesting a specific load balancer or when explicitly requested.

stats

object

Optional usage and rate-limit statistics for this load balancer over the last 24 hours.

created_at

string

The Unix timestamp (in seconds) when the load balancer was created.

updated_at

string

The Unix timestamp (in seconds) when the load balancer was last updated.

400 -

401 -

404 -

500 -

Update Load Balancer

PUT /ml/gateway/v1/load_balancers/{load_balancer_uuid}

Updates an existing user-defined load balancer by ID.

Request

Request Body

LoadBalancersUpdateLoadBalancerRequest

Load Balancer update request

name

string

A new name for the load balancer.

alias

string

A new alias for the load balancer.

algorithm

string

The load balancing algorithm to use for distributing requests.

Supported algorithms: "least_connections", "weighted_round_robin", "round_robin", "quota_priority"

config

object

Algorithm-specific configuration parameters. The structure of this object depends on the selected algorithm.

backends

array

Optional full replacement list of backends. When provided, all existing backends are removed and replaced in a single transaction.

Response

200 - Updated Load Balancer

uuid

string

The unique identifier for the load balancer.

name

string

The name of the load balancer.

alias

string

The alias that clients use to reference this load balancer.

algorithm

string

The load balancing algorithm used for distributing requests.

config

object

Algorithm-specific configuration parameters as raw JSON.

is_user_defined

boolean

Indicates whether this load balancer was created by a user (true) or generated from model aliases (false).

backends

array

The list of backends associated with this load balancer. This field is populated when requesting a specific load balancer or when explicitly requested.

stats

object

Optional usage and rate-limit statistics for this load balancer over the last 24 hours.

created_at

string

The Unix timestamp (in seconds) when the load balancer was created.

updated_at

string

The Unix timestamp (in seconds) when the load balancer was last updated.

400 -

401 -

404 -

500 -

Delete Load Balancer

DELETE /ml/gateway/v1/load_balancers/{load_balancer_uuid}

Removes a user-defined load balancer by ID.

Request

Response

204 - No Content

400 -

401 -

404 -

500 -

List Backends

GET /ml/gateway/v1/load_balancers/{load_balancer_uuid}/backends

Lists all backends for the specified load balancer.

Request

Response

200 - List of Backends

object

string

The object type, always "list".

data

array

The list of load balancer backends.

401 -

404 -

500 -

Create Backend

POST /ml/gateway/v1/load_balancers/{load_balancer_uuid}/backends

Creates a new backend for the specified load balancer.

Request

Request Body

LoadBalancersBackendPrototype

Backend configuration

model_uuid

Required *

string

The UUID of the model to use as a backend (required).

priority

integer

Priority determines the evaluation order for quota-priority routing (lower values have higher priority).

quota

integer

Optional quota for quota-priority strategy specifying the maximum number of concurrent connections.

weight

integer

The weight for this backend in weighted load balancing algorithms. Must be a positive integer.

Response

201 - Created Load Balancer Backend

id

string

The unique identifier for the backend.

load_balancer_id

string

The unique identifier of the load balancer this backend belongs to.

model_uuid

string

The UUID of the model used by this backend.

model_id

string

The model identifier as specified by the provider.

provider_name

string

The type of provider that hosts this model.

alias

string

The alias of the model used by this backend.

status

string

The status of the load balancer backend. Can be "active" or "unhealthy".

priority

integer

The priority order used during quota-priority routing (lower values evaluated first).

quota

integer

The quota for this backend when quota-priority strategy is used. Zero means unlimited.

weight

integer

The weight of this backend for weighted load balancing algorithms.

created_at

string

The Unix timestamp (in seconds) when the backend was created.

updated_at

string

The Unix timestamp (in seconds) when the backend was last updated.

request_count_24h

integer

The number of requests served by this model in the last 24 hours.

stats

object

Additive 24-hour usage and effective rate-limit metadata for this backend.

400 -

401 -

404 -

409 -

500 -

Get Load Balancer Backend

GET /ml/gateway/v1/load_balancers/{load_balancer_uuid}/backends/{load_balancer_backend_uuid}

Retrieves a specific load balancer backend by ID from a load balancer.

Request

Response

200 - Load Balancer Backend Configuration

id

string

The unique identifier for the backend.

load_balancer_id

string

The unique identifier of the load balancer this backend belongs to.

model_uuid

string

The UUID of the model used by this backend.

model_id

string

The model identifier as specified by the provider.

provider_name

string

The type of provider that hosts this model.

alias

string

The alias of the model used by this backend.

status

string

The status of the load balancer backend. Can be "active" or "unhealthy".

priority

integer

The priority order used during quota-priority routing (lower values evaluated first).

quota

integer

The quota for this backend when quota-priority strategy is used. Zero means unlimited.

weight

integer

The weight of this backend for weighted load balancing algorithms.

created_at

string

The Unix timestamp (in seconds) when the backend was created.

updated_at

string

The Unix timestamp (in seconds) when the backend was last updated.

request_count_24h

integer

The number of requests served by this model in the last 24 hours.

stats

object

Additive 24-hour usage and effective rate-limit metadata for this backend.

400 -

401 -

404 -

500 -

Replace Load Balancer Backend

PUT /ml/gateway/v1/load_balancers/{load_balancer_uuid}/backends/{load_balancer_backend_uuid}

Replaces a load balancer backend's configuration (currently only weight).

Request

Request Body

LoadBalancersUpdateBackendRequest

Replacement Load Balancer Backend Configuration

priority

integer

Optional priority override for quota-priority routing (smaller numbers evaluated first).

quota

integer

Optional quota override for quota-priority strategy.

weight

integer

The new weight for this backend in weighted load balancing algorithms. Must be a positive integer.

Response

200 - Updated Backend

id

string

The unique identifier for the backend.

load_balancer_id

string

The unique identifier of the load balancer this backend belongs to.

model_uuid

string

The UUID of the model used by this backend.

model_id

string

The model identifier as specified by the provider.

provider_name

string

The type of provider that hosts this model.

alias

string

The alias of the model used by this backend.

status

string

The status of the load balancer backend. Can be "active" or "unhealthy".

priority

integer

The priority order used during quota-priority routing (lower values evaluated first).

quota

integer

The quota for this backend when quota-priority strategy is used. Zero means unlimited.

weight

integer

The weight of this backend for weighted load balancing algorithms.

created_at

string

The Unix timestamp (in seconds) when the backend was created.

updated_at

string

The Unix timestamp (in seconds) when the backend was last updated.

request_count_24h

integer

The number of requests served by this model in the last 24 hours.

stats

object

Additive 24-hour usage and effective rate-limit metadata for this backend.

400 -

401 -

404 -

500 -

Delete Load Balancer Backend

DELETE /ml/gateway/v1/load_balancers/{load_balancer_uuid}/backends/{load_balancer_backend_uuid}

Removes a load balancer backend from the specified load balancer.

Request

Response

204 - Load Balancer Backend Deleted

400 -

401 -

404 -

500 -

List Rate Limit Configurations

GET /ml/gateway/v1/rate_limits

Lists all rate limit configurations for the tenant making the request.

Request

Response

200 - List of rate limit configurations

object

string

The object type, which is always `"list"`.

data

array

The list of objects.

has_more

boolean

Whether there are more items in the list.

next_page

string

The URL of the next page of results.

401 -

500 -

Create Rate Limit Configuration

POST /ml/gateway/v1/rate_limits

Creates a new rate limit configuration for the tenant making the request.

Request

Request Body

RateLimitPrototype

Rate limit Configuration

Response

201 - Rate limit created successfully

uuid

string

Unique identifier for the rate limit.

type

string

The type of rate limit, in this case should always be `"model"`.

request

RateLimitItem

Request rate limiting settings (per request origin).

token

RateLimitItem

Request rate limiting settings (per request origin).

provider_uuid

string

Unique identifier for the provider being rate limited.

model_uuid

string

Unique identifier for the model being rate limited.

400 -

401 -

500 -

Get Rate Limit Configuration

GET /ml/gateway/v1/rate_limits/{ratelimit_uuid}

Retrieves the current rate limit configuration for the tenant making the request.

Request

Response

200 - Rate limit configuration retrieved successfully

uuid

string

Unique identifier for the rate limit.

type

string

The type of rate limit, in this case should always be `"model"`.

request

RateLimitItem

Request rate limiting settings (per request origin).

token

RateLimitItem

Request rate limiting settings (per request origin).

provider_uuid

string

Unique identifier for the provider being rate limited.

model_uuid

string

Unique identifier for the model being rate limited.

401 -

404 -

500 -

Replace Rate Limit Configuration

PUT /ml/gateway/v1/rate_limits/{ratelimit_uuid}

Replaces the rate limit configuration for the tenant making the request.

Request

Request Body

RateLimitPrototype

Replacement Rate Limit Configuration

Response

200 - Rate Limit Configuration Replaced

uuid

string

Unique identifier for the rate limit.

type

string

The type of rate limit, in this case should always be `"model"`.

request

RateLimitItem

Request rate limiting settings (per request origin).

token

RateLimitItem

Request rate limiting settings (per request origin).

provider_uuid

string

Unique identifier for the provider being rate limited.

model_uuid

string

Unique identifier for the model being rate limited.

400 -

401 -

404 -

500 -

Delete Rate Limit Configuration

DELETE /ml/gateway/v1/rate_limits/{ratelimit_uuid}

Deletes the rate limit configuration identified by the provided UUID for the tenant making the request.

Request

Response

204 - Rate Limit Configuration Deleted

401 -

404 -

500 -

Create Response

POST /ml/gateway/v1/responses

Creates a model response using the OpenAI-compatible Responses API.

Request

Request Body

CreateResponseRequest

Create Response Request

model

Required *

string

The model to use for generating the response

input

object

The input for response generation. Provide either a plain string or an array of input items.

instructions

string

Instructions for the model

max_output_tokens

integer

Maximum number of tokens to generate

max_tool_calls

integer

Maximum number of tool calls allowed

temperature

number

Sampling temperature

top_p

number

Nucleus sampling parameter

frequency_penalty

number

Frequency penalty

presence_penalty

number

Presence penalty

stream

boolean

Whether to stream the response

stream_options

StreamOptions

Options for streaming response. Only set this when you set `stream` to `true`.

include_usage

boolean

If set, an additional chunk will be streamed before the data: [DONE] message. The usage field on this chunk shows the token usage statistics for the entire request, and the choices field will always be an empty array. All other chunks will also include a usage field, but with a null value.

store

boolean

Whether to store the response

metadata

object

Additional metadata

tools

array

Tools available for the model

tool_choice

object

Tool choice configuration

parallel_tool_calls

boolean

Whether to allow parallel tool calls

reasoning

object

Reasoning configuration

text

object

Text configuration

background

boolean

Whether to run in background

include

array

Fields to include in response

previous_response_id

string

ID of previous response to continue from

prompt_cache_key

string

Cache key for prompt

safety_identifier

string

Safety identifier

service_tier

string

Service tier

top_logprobs

integer

Number of top log probabilities to return

truncation

string

Truncation strategy

Response

200 - Successful Response

id

string

Unique identifier for the response

object

string

Object type

created_at

integer

Unix timestamp of when the response was created

model

string

The model used for the response

status

string

Status of the response (e.g., completed, in_progress, failed)

output

array

The generated output items

completed_at

integer

Unix timestamp of when the response was completed

error

object

Error information if the response failed

incomplete_details

object

Details about why the response is incomplete

instructions

string

Instructions used for the model

max_output_tokens

integer

Maximum number of tokens to generate

max_tool_calls

integer

Maximum number of tool calls allowed

temperature

number

Sampling temperature used

top_p

number

Nucleus sampling parameter used

frequency_penalty

number

Frequency penalty used

presence_penalty

number

Presence penalty used

metadata

object

Additional metadata

tools

array

Tools that were available

tool_choice

object

Tool choice configuration used

parallel_tool_calls

boolean

Whether parallel tool calls were allowed

reasoning

object

Reasoning configuration used

text

object

Text configuration used

background

boolean

Whether the response ran in background

previous_response_id

string

ID of previous response this continued from