Adding a grammar to a custom language model

Create a custom language model

To use a grammar with speech recognition, you must add it to a custom language model. You can use an existing custom language model, or you can create a new custom model by using the POST /v1/customizations method. For information about creating a new custom model, see Create a custom language model.

A custom language model can contain corpora and custom words as well as grammars. During speech recognition, you can use the custom model with or without its grammars. However, when you use a grammar, the service recognizes only words from the specified grammar.

Add a grammar to the custom language model

You use the POST /v1/customizations/{customization_id}/grammars/{grammar_name} method to add a grammar file to a custom model.

customization_id (required string)

Specify the customization ID of the custom model to which the grammar is to be added.

grammar_name (required string)

Specify a name for the grammar. Use a localized name that matches the language of the custom model and reflects the contents of the grammar.

• Include a maximum of 128 characters in the name.
• Do not use characters that need to be URL-encoded. For example, do not use spaces, slashes, backslashes, colons, ampersands, double quotes, plus signs, equals signs, questions marks, and so on in the name. (The service does not prevent the use of these characters. But because they must be URL-encoded wherever used, their use is strongly discouraged.)
• Do not use the name of a grammar or corpus that has already been added to the custom model.
• Do not use the name user, which is reserved by the service to denote custom words that are added or modified by the user.
• Do not use the name base_lm or default_lm. Both names are reserved for future use by the service.

Content-Type (required string)

Use the request header to specify the format of the grammar:

• application/srgs for an ABNF grammar
• application/srgs+xml for an XML grammar

Pass the grammar text file as the body of the request. The following example adds the grammar file named confirm.abnf to the custom model with the specified ID. The example names the grammar confirm-abnf.

IBM Cloud

curl -X POST -u "apikey:{apikey}" \
--header "Content-Type: application/srgs" \
--data-binary @confirm.abnf \
"{url}/v1/customizations/{customization_id}/grammars/confirm-abnf"

IBM Cloud Pak for Data

curl -X POST \
--header "Authorization: Bearer {token}" \
--header "Content-Type: application/srgs" \
--data-binary @confirm.abnf \
"{url}/v1/customizations/{customization_id}/grammars/confirm-abnf"

The method also accepts an optional allow_overwrite query parameter that you can include to overwrite an existing grammar with the same name. Use the parameter if you need to update a grammar after you add it to a model.

The method is asynchronous. It can take a few seconds or minutes for the service to analyze the grammar, depending on the size of the grammar and the current load on the service. For information about checking the status of a grammar, see Monitoring the add grammar request.

You can add any number of grammars to a custom model by calling the method once for each grammar file. The addition of one grammar must be fully complete before you can add another.

curl -X GET -u "apikey:{apikey}" \
"{url}/v1/customizations/{customization_id}/grammars/confirm-abnf"

curl -X GET \
--header "Authorization: Bearer {token}" \
"{url}/v1/customizations/{customization_id}/grammars/confirm-abnf"

{
  "name": "confirm-abnf",
  "out_of_vocabulary_words": 0,
  "status": "being_processed"
}

{
  . . .
  "error": "{\"code\":500, \"code_description\":\"Internal Server Error\",
\". . . Cannot compile grammar . . .\"},
  . . .
}

Validate the words from the grammar

When you add a grammar file to a custom language model, the service parses the grammar to determine whether the grammar recognizes any words that are not already part of the service's base vocabulary. Such words are referred to as out-of-vocabulary (OOV) words. The service adds OOV words to the custom model's words resource. The purpose of the words resource is to define words that are not already present in the service's vocabulary.

Definitions in the words resource tell the service how to transcribe the OOV words. The information includes a sounds_like field that tells the service how the word is pronounced, a display_as field that tells the service how to display the word, and a source field that indicates how the word was added to the custom model. For more information about the words resource and OOV words, see The words resource.

After you add a grammar to a custom model, it is good practice to examine the OOV words in the model's words resource to verify their sounds-like pronunciations. Not all grammars have OOV words, but verifying the words resource is generally a good idea. To check the words of the custom model after you add a grammar, use the following methods:

• Use the GET /v1/customizations/{customization_id}/words method to list all of the words from a custom model. Pass the value grammars with the method's word_type parameter to list only words that were added from grammars.
• Use the GET /v1/customizations/{customization_id}/words/{word_name} method to view an individual word from a model.

Verify that the sounds-like pronunciations of the words are accurate and correct. Also look for typographical and other errors in the words themselves. For more information about validating and correcting the words in a custom model, see Validating a words resource for previous-generation models.

Train the custom language model

The final step before you can use a grammar with a custom language model is to train the model. You use the same process that you use for training a custom model on new corpora or new custom words. Training the model compiles the grammar for use during speech recognition. The service processes the grammar from its original text-based format to a binary runtime format for speech recognition. You cannot use the grammar until you train the model.

You use the POST /v1/customizations/{customization_id}/train method to train a custom model. You pass the method the customization ID of the model that you want to train, as in the following example.

IBM Cloud

curl -X POST -u "apikey:{apikey}" \
"{url}/v1/customizations/{customization_id}/train"

IBM Cloud Pak for Data

curl -X POST \
--header "Authorization: Bearer {token}" \
"{url}/v1/customizations/{customization_id}/train"

The training method is asynchronous. Training typically takes only seconds or a few minutes to complete. But it can take longer depending on the size and complexity of the grammar and the current load on the service. For information about checking the status of a training operation, see Monitoring the training request.

{
  "customization_id": "74f4807e-b5ff-4866-824e-6bba1a84fe96",
  "created": "2018-06-01T18:42:25.324Z",
  "language": "en-US",
  "dialect": "en-US",
  "owner": "297cfd08-330a-22ba-93ce-1a73f454dd98",
  "name": "Example model",
  "description": "Example custom language model",
  "base_model_name": "en-US_BroadbandModel",
  "status": "training",
  "progress": 0
}