Elasticsearch search integration setup

Using an existing index

1. In the Select index section, click Use my index to connect to an existing Elasticsearch index. The Use my index option is selected as default in your Elasticsearch set up.

2. In the Use my index option, type the Elasticsearch index name.

3. Click Next to go to the Enable conversational search (optional) section.

4. Conversational search is available only in the Plus and Enterprise plans of watsonx Assistant. In the Enable conversational search (optional) section, switch the Conversational Search toggle to on if you want to activate conversational search. If you don't want to activate conversational search, switch the toggle to off. For more information about conversational search, see conversational search.

5. Click Save and then Close.

Uploading documents to a new index Beta

Before you upload documents, your Elasticsearch instance must have the following prerequisites:

• Elasticsearch 8.8 or above.

• A paid or trial subscription of the Elasticsearch instance such as the Platinum Edition of IBM Cloud Databases for Elasticsearch or the Platinum or Enterprise subscription that is offered by Elastic.co.

• A Machine Learning (ML) node with a minimum of 4 GB memory to deploy the ELSER model. For more information about the ELSER requirements, see ELSER requirements.

• The documents that you upload must be in English.

If your Elasticsearch instance does not have the prerequisites for uploading documents, you see the Requirements not met error message.

If you experience a delay or failure in uploading documents even after meeting the prerequisites, you can consider scaling the inference performance of the ELSER model deployment by setting up parameters such as number_of_allocations and threads_per_allocation. For more information about scaling the inference performance, see Start trained model deployment API.

To upload documents to a new index, use the following procedure:

1. In the Select index section of the Elasticsearch window, click Upload documents to a new index in your Elasticsearch instance.

   watsonx Assistant passes the uploaded documents to your Elasticsearch instance for storage, chunking, and indexing.

2. In the Configure result content section, provide the following fields to map the title, body, and URL to the search response:

   • Title

     Search result title. Use the title, name, or similar type of field from the collection as the search result title.

     You must select something for the title or no search result response is displayed in the Facebook and Slack integrations.

   • Body

     Search result description. Use an abstract, summary, or highlight field from the collection as the search result body.

     You must select something for the body or no search result response is displayed in the Facebook and Slack integrations.

   • URL

     This field can be populated with any footer content that you want to include at the end of the search result.

   When you configure the query body in the Advanced Elasticsearch Settings to search the nested documents, you must ensure that the Title, Body, and URL are from the fields of the inner documents in your Elasticsearch index. For more information about using nested queries, see Elasticsearch nested query.

3. Expand the Advanced Elasticsearch settings section to see the following text boxes:

   • Configure the filter array for Elasticsearch

     You define the filter as an array of objects so that you can create filters to arrange the content per the query body. For more information, see Configuring the custom filters.

   • Configure the query body for Elasticsearch

     The query body is used to manipulate the user requests into a format that the search expects. It controls the query forms, search fields, filters, and query size. In the REST API, the query body is an object that represents the POST body for the _search request to Elasticsearch. THe query body has a "$QUERY" token to represent the customer's query, and a "$FILTER" token to represent the array of filters that are defined either in the search settings or at the step level.

     By default, Elasticsearch integration uses keyword search. You can use advanced search such as semantic search with ELSER, KNN dense vector search, nested queries to search the nested documents, or hybrid search. For more information about using different types of query body examples, see Query body examples.

     For more information about the Elasticsearch _search API request body, see Elasticsearch search API request body.

     You cannot customize the query body in the assistant with an existing Elasticsearch configuration.

4. Use the Message, No results found and Connectivity issue tabs to customize different messages to share with users based on the successfulness of the search.

   Search result messages

   Tab	Scenario	Example message
   Message	Search results are returned	I found this information that might be helpful:
   No results found	No search results are found	I searched my knowledge base for information that might address your query, but did not find anything useful to share.
   Connectivity issue	I was unable to complete the search for some reason	I might have information that could help address your query, but am unable to search my knowledge base at the moment.

5. Switch the Conversational Search toggle to on if you want to activate conversational search. If you don't want to activate conversational search, switch the toggle to off.

   Conversational search is available only in the Plus and Enterprise plans of watsonx Assistant. If you switch the Conversational Search toggle to on, you can see the citation titles in your assistant responses. For more information about conversational search, see conversational search.

6. Click Save to save your settings.

7. Click the Documents tab in the Elasticsearch window.

   The Documents tab is enabled only if you select Upload documents to a new index in your Elasticsearch instance option.

8. Click Upload button. In the Upload documents section, you can drag your files or do a single click to upload documents directly to your assistant.

   You can upload up to 20 documents at a time. Each document file size must not exceed 25 KB. The total size of all documents must not exceed 50 MB.

   

9. After you upload the documents, you can see the upload status of your documents in a table in the Elasticsearch window.

10. Status Ready indicates that your files are available for search.

11. If the status indicates Error, you can delete the file by clicking the three dots next to the Error and click Delete.

    

12. Skip this step if you do not want to change Elasticsearch instance credentials. If you want to change the Elasticsearch instance credentials, click the Instance tab, edit the credentials, and then click Save.

13. Click Save and then Close to end the Elasticsearch set up.