Create a chat completion
POST /inference/chat/completions
Creates a model response for the given chat conversation. This method accepts a chat conversation and generates a response from the specified model. The request must include the conversation history and model identifier, and returns the generated chat completion.
Authorization
To call this method, you must be assigned one or more IAM access roles that include the following action.
instructlab.inference.create
Auditing
Calling this method generates the following auditing event.
instructlab.inference.create
Request
Request Body
OpenAIChatCompletionRequestWithExtraBody
model
Required *
string
messages
Required *
array
frequency_penalty
number
function_call
object
functions
array
logit_bias
object
logprobs
boolean
max_completion_tokens
integer
max_tokens
integer
n
integer
parallel_tool_calls
boolean
presence_penalty
number
response_format
object
seed
integer
stop
object
stream
boolean
stream_options
object
temperature
number
tool_choice
object
tools
array
top_logprobs
integer
top_p
number
user
string
Response
201 -
400 -
401 -
403 -
405 -
500 -
List chat completions
GET /inference/chat/completions
Lists all stored chat completions that have been created in your project. This method retrieves a collection of previously generated chat completions, allowing you to review past inference requests and their results.
Authorization
To call this method, you must be assigned one or more IAM access roles that include the following action.
instructlab.inference.list
Auditing
Calling this method generates the following auditing event.
instructlab.inference.list
Request
Response
200 -
400 -
401 -
403 -
405 -
500 -
Get a chat completion
GET /inference/chat/completions/{id}
Retrieves a specific stored chat completion by its unique identifier. This method returns the complete details of a previously generated chat completion, including the original request parameters and the model's response.
Authorization
To call this method, you must be assigned one or more IAM access roles that include the following action.
instructlab.inference.get
Auditing
Calling this method generates the following auditing event.
instructlab.inference.get
Request
Path Parameters
id
Required *
string
The ID of the chat completion to retrieve.
Query Parameters
Response
200 -
400 -
401 -
403 -
405 -
500 -
Delete a chat completion
DELETE /inference/chat/completions/{id}
Deletes a specific stored chat completion from your project. This method permanently removes the chat completion record and its associated data.
Authorization
To call this method, you must be assigned one or more IAM access roles that include the following action.
instructlab.inference.delete
Auditing
Calling this method generates the following auditing event.
instructlab.inference.delete
Request
Path Parameters
id
Required *
string
The ID of the chat completion to delete.
Query Parameters
Response
200 -
400 -
401 -
403 -
405 -
500 -
Create an embedding
POST /inference/embeddings
Creates an embedding vector for the input text by using the specified model. The resulting embeddings can be used for semantic search, clustering, and other machine learning tasks.
Authorization
To call this method, you must be assigned one or more IAM access roles that include the following action.
instructlab.inference-embeddings.create
Auditing
Calling this method generates the following auditing event.
instructlab.inference-embeddings.create
Request
Request Body
CreateEmbeddingRequest
input
Required *
EmbeddingInput
Input text to embed, encoded as a string or array of tokens. To embed multiple inputs in a single request, pass an array of strings or array of token arrays. The input must not exceed the maximum input tokens for the model (8192 tokens for `text-embedding-ada-002`) and cannot be an empty string. Any array must contain 2,048 items or fewer.
model
Required *
string
The ID of the model to use for generating embeddings.
encoding_format
string
The format in which the embeddings are returned. Accepted values are `float` and `base64`.
dimensions
integer
The number of dimensions in the output embedding vectors. Only `text-embedding-3` and newer models support this parameter.
user
string
A unique identifier for the end user. Providing this value enables monitoring and detection of misuse.
Response
201 -
400 -
401 -
403 -
405 -
500 -
List models
GET /inference/models
Lists the currently available models and provides basic information about each one, such as the owner and availability.
Authorization
To call this method, you must be assigned one or more IAM access roles that include the following action.
instructlab.inference-models.list
Auditing
Calling this method generates the following auditing event.
instructlab.inference-models.list
Request
Response
200 -
400 -
401 -
403 -
405 -
500 -
Get a model
GET /inference/models/{model}
Retrieves detailed information about a specific model. This method provides basic information about the model, including the owner and permissioning details.
Authorization
To call this method, you must be assigned one or more IAM access roles that include the following action.
instructlab.inference-model.get
Auditing
Calling this method generates the following auditing event.
instructlab.inference-model.get
Request
Path Parameters
model
Required *
string
The ID of the model to use for this request