Configure AI model for Retrieval Service
This configuration is available only with the Gen AI experience.
Configure the foundation model that powers retrieval-based tasks in watsonx.data.
About this task
The Retrieval Service in watsonx.data enables administrators to configure which foundation model powers retrieval-based tasks such as text-to-SQL, question answering, and RAG. At the instance level, you can choose between granite (default), llama and gpt models, based on licensing and workload requirements
Configuring the retrieval service model
-
Log in to watsonx.data console.
-
From the navigation menu, select Configurations and click Retrieval service model tile.
-
Under Retrival service section, choose one of the following available AI models:
- granite-3-8b-instruct
- llama-3-3-70b-instruct
- gpt-oss-120b
To use gpt-oss-120b with the Retrieval Service, you must first deploy the model in toronto region. For detailed instructions, see Deploying foundation models on demand (fast path). -
A confirmation dialog appears, click Select.
Configuring the Text to SQL model
- In the Text to SQL section, select the AI model to use for Text to SQL conversion:
- granite-3-8b-instruct
- llama-3-3-70b-instruct
- A confirmation dialog appears, click Select.