Before you begin

• Create a Pay-As-You-Go or Subscription IBM Cloud account. Trial accounts are not supported. For more information or to upgrade your account, see Account types.

• Create a Red Hat AI Inference project.

• Make sure that you have the Writer role or greater on the Red Hat AI Inference service. For more information, see Managing IAM access.

Before you begin

• Create a Pay-As-You-Go or Subscription IBM Cloud account. Trial accounts are not supported. For more information or to upgrade your account, see Account types.

• Create a Red Hat AI Inference project.

• Make sure that you have the Writer role or greater on the Red Hat AI Inference service. For more information, see Managing IAM access.

• Authenticate to the API.

• Get your API endpoint. All API requests use the following base URL format:

  https://us-east.rhai.ibm.com/v1/projects/<project_id>/inference
  

  Replace <project_id> with your project ID. To find it, go to Red Hat AI Inference projects and open your project.

Listing models by using the console

The console provides an interactive playground where you can experiment with different models, test prompts, and refine your AI interactions before integrating them into your applications.

The playground does not support embeddings. Use the API to create embeddings.

1. In the console, open the Red Hat AI Inference service.
2. Click Model catalog to explore the available models.

Listing models by using the API

The following example shows how to list models. For a complete list of the available parameters, see OpenAI List Models.

curl -L 'https://us-east.rhai.ibm.com/v1/projects/<project_id>/inference/models' \
-H 'Accept: application/json' -H "Authorization: Bearer <bearer_token>"

from openai import OpenAI
client = OpenAI(
  api_key="<bearer_token>",
  base_url="https://us-east.rhai.ibm.com/v1/projects/<project_id>/inference",
)

models = client.models.list()
print(models)

If supports_embeddings is true in the model response, the model can be used to create embeddings.

Getting a model by ID by using the API

Retrieving detailed information about a specific model helps you understand its characteristics, capabilities, and limitations before using it in your application.

The following example shows how to get a model by ID. For a complete list of the available parameters, see Retrieve Model.

curl -L 'https://us-east.rhai.ibm.com/v1/projects/<project_id>/inference/models/<model>' \
-H 'Accept: application/json' -H "Authorization: Bearer <bearer_token>"

from openai import OpenAI
client = OpenAI(
  api_key="<bearer_token>",
  base_url="https://us-east.rhai.ibm.com/v1/projects/<project_id>/inference",
)

model = client.models.retrieve("<model>")  # for example, "granite-4-0-h-small"
print(model)

If supports_embeddings is true in the model response, the model can be used to create embeddings.