Handling phone interactions

Adding phone-specific responses to your assistant

To initiate a voice-specific interaction from an action step, add a response in the generic array with the appropriate response type. For more information about using the JSON editor to add responses, see Defining responses with the JSON editor.

Applying advanced settings to the Speech to Text service

Use the speech_to_text response type to send configuration commands to the Speech to Text service instance used by the phone integration. By sending a speech_to_text response from an action step, you can dynamically change the Speech to Text configuration during a conversation.

By default, any Speech to Text configuration changes you make persist for the remainder of the conversation, or until you update them again. You can change this behavior by specifying the update_strategy property of the parameters object.

The format of the speech_to_text response type is as follows:

{
  "generic": [
    {
      "response_type": "speech_to_text",
      "command_info": {
        "type": "<command type>",
        "parameters": {
          "parameter name1": "parameter value",
          "parameter name2": "parameter value"
        }
      }
    }
  ]
}

Each command type along with its related parameters are described in the following sections.

{
  "generic": [
    {
      "response_type": "speech_to_text",
      "command_info": {
        "type": "configure",
        "parameters": {
          "narrowband_recognize": {
            "x-watson-learning-opt-out": true,
            "model": "en-US_NarrowbandModel",
            "profanity_filter": true,
            "smart_formatting": true,
            "language_customization_id": "81d3630-ba58-11e7-aa4b-41bcd3f6f24d",
            "acoustic_customization_id": "e4766090-ba51-11e7-be33-99bd3ac8fa93"
          }
        }
      }
    }
  ]
}

{
  "generic": [
    {
      "response_type": "speech_to_text",
      "command_info": {
        "type": "configure",
        "parameters": {
          "update_strategy": "merge_once",
          "narrowband_recognize": {
            "x-watson-learning-opt-out": true,
            "grammar_name": "names-abnf",
            "language_customization_id": "81d3630-ba58-11e7-aa4b-41bcd3f6f24d"
          }
        }
      }
    }
  ]
}

{
  "generic": [
    {
      "response_type": "speech_to_text",
      "command_info": {
        "type": "configure",
        "parameters": {
          "narrowband_recognize": {
            "model": "es-ES_NarrowbandModel",
            "smart_formatting": true
          }
        }
      }
    }
  ]
}

{
  "generic": [
    {
      "response_type": "speech_to_text",
      "command_info": {
        "type": "configure",
        "parameters": {
          "update_strategy": "merge_once",
          "narrowband_recognize": {
            "language_customization_id": "ao45vohgFuxyOQRgztu-02I10ut7aJcM-AdInT-VWgj3V" 
          }
        }
      }
    }
  ]
}

Applying advanced settings to the Text to Speech service

Use the text_to_speech response type to send configuration commands to the Text to Speech service instance used by the phone integration. By sending a text_to_speech response from an action step, you can dynamically change the Text to Speech configuration during a conversation.

By default, any Text to Speech configuration changes you make persist for the remainder of the conversation, or until you update them again. You can change this behavior by specifying the update_strategy property of the parameters object.

The format of the speech_to_text response type is as follows:

{
  "generic": [
    {
      "response_type": "text_to_speech",
      "command_info": {
        "type": "<command type>",
        "parameters": {
          "parameter name": "parameter value",
          "parameter name": "parameter value"
        }
      }
    }
  ]
}

Each command type along with its related parameters are described in the following sections.

{
  "generic": [
    {
      "response_type": "text_to_speech",
      "command_info": {
        "type": "configure",
        "parameters": {
          "synthesize": {
            "voice": "en-GB_KateV3Voice"
          }
        }
      }
    }
  ]
}

Transferring a call to a live agent

When you configure the phone integration, you can optionally set up backup call center support, which makes it possible for the assistant to transfer a call to a human. You can use the Connect to agent response type in an action step to initiate a transfer to a live agent at a specific point in the conversation. When a Connect to agent response is sent to the phone integration, a SIP transfer is initiated with the SIP REFER message, as defined by RFC 5589.

For more information about initiating a transfer to a live agent during the conversation, see the following documentation:

• Setting up live agent escalation
• Defining responses with the JSON editor

The phone integration supports more parameters for the Connect to agent response type. You can add these phone-specific parameters to the connect_to_agent response type with the JSON editor.

The connect_to_agent response type supports the ability to specify the target transfer information under the transfer_info parameter.

The following example shows a transfer that uses all of the configurable parameters:

{
  "generic": [
    {
      "response_type": "connect_to_agent",
      "transfer_info": {
        "target": {
          "service_desk": {
            "sip": {
              "uri": "sip:user\\@domain.com",
              "transfer_headers": [
                {
                  "name": "Customer-Header1",
                  "value": "Some-Custom-Info"
                },
                {
                  "name": "User-to-User",
                  "value": "XXXXXX"
                }
              ],
              "transfer_headers_send_method": "refer_to_header"
            }
          }
        }
      },
      "agent_available": {
        "message": "I'll transfer you to an agent"
      },
      "agent_unavailable": {
        "message": "Sorry, I could not find an agent."
      },
      "message_to_human_agent": "The caller needs help resetting their password"
    }
  ]
}

The connect_to_agent response type supports the following phone-specific properties.

If you define a SIP URI as the transfer target, escape the at sign (@) in the URI by adding two backslashes (\\) in front of it. This prevents the string from being recognized as part of the entity shorthand syntax.

    "uri": "sip:12345556789\\@myhost.com"

{
  "generic": [
    {
      "response_type": "connect_to_agent",
      "transfer_info": {
        "target": {
          "service_desk": {
            "sip": {
              "uri": "sip:user\\@domain.com",
              "transfer_method": "hangup",
              "transfer_target_header": "Transfer-Target"
            }
          }
        }
      },
      "agent_available": {
        "message": "Please hold on while I connect you with a live agent."
      },
      "agent_unavailable": {
        "message": "Sorry, I could not find an agent."
      },
      "message_to_human_agent": "The caller needs help resetting their password"
    }
  ]
}

REFER sip:b@atlanta.example.com SIP/2.0
Via: SIP/2.0/UDP agenta.atlanta.example.com;branch=z9hG4bK2293940223
To: <sip:b@atlanta.example.com>
From: <sip:a@atlanta.example.com>;tag=193402342
Call-ID: 898234234@agenta.atlanta.example.com
CSeq: 23 REFER
Max-Forwards: 7
Refer-To: sip:user@domain.com
X-Watson-Assistant-Token: 8f817472-8c57-4117-850d-fdf4fd23ba7
User-to-User: 637573746f6d2d757365722d746f2d75736572;encoding=hex
Contact: sip:a@atlanta.example.com
Content-Length: 0

REFER sip:b@atlanta.example.com SIP/2.0
Via: SIP/2.0/UDP agenta.atlanta.example.com;branch=z9hG4bK2293940223
To: <sip:b@atlanta.example.com>
From: <sip:a@atlanta.example.com>;tag=193402342
Call-ID: 898234234@agenta.atlanta.example.com
CSeq: 93809823 REFER
Max-Forwards: 70
Refer-To: sip:user@domain.com
User-to-User: 637573746f6d2d757365722d746f2d75736572;encoding=hex
X-Watson-Assistant-Session-History-Key: dev::latest::212033::0a64c30d-c558-4055-85ad-ef75ad6cc29d::978f1fd7-4e24-47d8-adb0-24a8a6eff69e::b5ffd6c2-902f-4658-b586-e3fc170a6cf3::7ad616a350cc48078f17e3ee3df551de
Contact: sip:a@atlanta.example.com
Content-Length: 0

REFER sip:b@atlanta.example.com SIP/2.0
Via: SIP/2.0/UDP agenta.atlanta.example.com;branch=z9hG4bK2293940223
To: <sip:b@atlanta.example.com>
From: <sip:a@atlanta.example.com>;tag=193402342
Call-ID: 898234234@agenta.atlanta.example.com
CSeq: 23 REFER
Max-Forwards: 70
Refer-To: sip:user@domain.com?User-to-User=637573746f6d2d757365722d746f2d75736572%3Bencoding%3Dhex
Contact: sip:a@atlanta.example.com
Content-Length: 0

REFER sip:b@atlanta.example.com SIP/2.0
Via: SIP/2.0/UDP agenta.atlanta.example.com;branch=z9hG4bK2293940223
To: <sip:b@atlanta.example.com>
From: <sip:a@atlanta.example.com>;tag=193402342
Call-ID: 898234234@agenta.atlanta.example.com
CSeq: 93809823 REFER
Max-Forwards: 70
Refer-To: sip:user@domain.com?User-to-User=637573746f6d2d757365722d746f2d75736572%3Bencoding%3Dhex&X-Watson-Assistant-Session-History-Key=dev::latest::893499::dff9c274-adc4-4f63-93de-781166760bf8::978f1fd7-4e24-47d8-adb0-24a8a6eff69e::b5ffd6c2-902f-4658-b586-e3fc170a6cf3::7ad616a350cc48078f17e3ee3df551de
Contact: sip:a@atlanta.example.com
Content-Length: 0

Playing hold music or a voice recording

To play hold music or to play a recorded message, use the audio response type. For more information about response types, see Defining responses with the JSON editor.

You cannot play hold music during a call transfer. However, you might want to play hold music if your assistant needs time to perform processing of some kind, such as calling a client-side action or making a call to a webhook.

The phone integration supports the following properties for the audio response type:

If you set channel_options.voice_telephony.loop to true, add a user-defined response with the vgwActForceNoInputTurn command. This command instructs the phone integration to initiate a turn with a vgwNoInputTurn text without waiting for an input from the caller. In the vgwNoInputTurn turn you can initiate a transaction while the caller is on hold. When the vgwNoInputTurn turn completes, the looped audio stops.

The following example shows an audio response with loop=true, and a user_defined response with the vgwActForceNoInputTurn command.

{   
  "generic": [
    {
      "response_type": "user_defined",
      "user_defined": {
        "vgwAction": {
          "command": "vgwActForceNoInputTurn"
        }
      }
    },
    {
      "response_type": "audio",
      "source": "https://upload.wikimedia.org/wikipedia/commons/d/d8/Random_composition3.wav",
      "channel_options": {
        "voice_telephony": {
          "loop": true
        }
      }
    }
  ]
}

Enabling keypad entry

If you want customers to be able to send information by typing it on their phone keypad instead of speaking, you can add support for phone keypad entry. The best way to implement this type of support is to enable dual-tone multifrequency (DTMF) signaling. DTMF is a protocol to transmit tones that are generated when a user presses keys on a push-button phone. The tones have a specific frequency and duration that can be interpreted by the phone network.

To start listening for tones as the user presses phone keys, use the dtmf response type in an action step. This response type can be added by using the JSON editor.

{
  "generic": [
    {
      "response_type": "dtmf",
      "command_info": {
        "type": "<command type>",
        "parameters": {
          "parameter name": "parameter value",
          "parameter name": "parameter value"
        }
      },
      "channels": [
        {
          "channel": "voice_telephony"
        }
      ]
    }
  ]
}

The command_info property specifies a DTMF command for the phone integration. The supported commands and their related parameters are as follows.

{
  "generic": [
    {
      "response_type": "dtmf",
      "command_info": {
        "type": "collect",
        "parameters": {
          "termination_key": "#",
          "count": 16,
          "ignore_speech": true
        }
      },
      "channels": [
        {
          "channel": "voice_telephony"
        }
      ]
    }
  ]
}

{
  "generic": [
    {
      "response_type": "dtmf",
      "command_info": {
        "type": "send",
        "parameters": {
          "digits": [
            {
              "code": "9",
              "volume": -8
            },
            {
              "code": "11"
            }
          ],
          "send_interval": 100
        }
      },
      "channels": [
        {
          "channel": "voice_telephony"
        }
      ]
    }
  ]
}

Transferring the conversation to the web chat integration

You can transfer the caller from the current phone call to a web chat session by using the channel_transfer response type.

The assistant sends an SMS message to the caller that includes a URL that the caller can tap to load the web chat widget in the phone's browser. The web chat session displays the history of the phone call and can start the process of collecting information that is needed to complete the transaction.

This is useful in situations when the customer can provide information more easily in writing than by speaking (for example, changing an address).

After the transfer successfully completes, the caller can hang up the phone and continue the conversation by using web chat.

The channel_transfer response type can be used with the phone integration only if the SMS with Twilio integration is also configured for the assistant.

{
  "generic": [
    {
      "response_type": "text",
      "values": [
        {
          "text": "I will send you a text message now with a link to our website."
        }
      ],
      "selection_policy": "sequential"
    },
    {
      "response_type": "channel_transfer",
      "message_to_user": "Click the link to connect with an agent using our website.",
      "transfer_info": {
        "target": {
          "chat": {
            "url": "https://example.com/webchat"
          }
        }
      }
    }
  ]
}

Ending the call

You can instruct your assistant to end a phone call by using the end_session response type, as shown in this example.

{
  "generic": [
    {
      "response_type": "end_session"
    }
  ]
}

You can optionally include custom headers to include with the SIP BYE request that is generated when the phone integration receives this response type.

This example shows the end_session response type with custom SIP headers:

{
  "generic": [
    {
      "response_type": "end_session",
      "channel_options": {
        "voice_telephony": {
          "sip": {
            "headers": [
              {
                "name": "Customer-Header1",
                "value": "Some-Custom-Info"
              },
              {
                "name": "User-to-User",
                "value": "XXXXXX"
              }
            ]
          }
        }
      }
    }
  ]
}

Sending a text message during a phone conversation

In situations when it is easier to communicate accurately in writing than by transcribing voice output, you can send a text message during an ongoing voice. For example, an address or directions.

Before you can send SMS messages during a phone call, you must set up the SMS integration. For more information, see Integrating with SMS.

When you exchange a text with a customer during a conversation, the assistant initiates the SMS message exchange. A text message is sent to the user and asks for the user to respond.

To send a specific message from an action step, use the user_defined response type with the vgwActSendSMS command:

{
  "generic": [
    {
      "response_type": "text",
      "values": [
        {
          "text": "I will send you a text message now."
        }
      ],
      "selection_policy": "sequential"
    },
    {
      "response_type": "user_defined",
      "user_defined": {
        "vgwAction": {
          "command": "vgwActSendSMS",
          "parameters": {
            "message": "To send me your street address, respond to this text message with your address."
          }
        }
      }
    }
  ]
}

You can specify any of the following parameters in the parameters object:

If your SMS integration supports more than one SMS phone number, or you are using a SIP trunk different from your SMS provider, be sure to specify the phone number that you want to use to send the text message. Otherwise, the text is sent by using the same phone number that was called.

After the assistant receives an SMS message, a new conversation turn is initiated with the text input vgwSMSMessage. This input indicates that a message was received from the caller. The text of the customer's message is included as the value of the vgwSMSMessagecontext variable and the sms_message integration variable.

If the assistant is unable to send an SMS message to the caller, a new turn is initiated with the text input vgwSMSFailed. This input indicates that an SMS message could not be sent to the caller. You can design your assistant to handle such a failure by creating actions that are triggered by the input text vgwSMSFailed.

{
  "input": {
    "text": "vgwSMSMessage",
    "integrations": {
      "voice_telephony": {
        "sms_message": "230 Leigh Farm rd"
      }
    }
  },
  "context": {
    "skills": {
      "main skill": {
        "user_defined": {
          "vgwSMSMessage": "1545 Lexington Ave."
        }
      }
    }
  }
}

Here's an example of a turn request when setAsInputText is set to true:

{
  "input": {
    "text": "230 Leigh Farm rd",
    "integrations": {
      "voice_telephony": {
        "sms_message": "230 Leigh Farm rd"
      }
    }
  },
  "context": {
    "skills": {
      "main skill": {
        "user_defined": {
          "vgwSMSMessage": "1545 Lexington Ave."
        }
      }
    }
  }
}

Defining a sequence of phone commands

If you want to run more than one command in succession, include multiple responses in the generic array. These commands are processed in the order in which they are specified in the array.

This example shows two responses: first, a text response; and second, an end_session response to end the call.

{
  "generic": [
    {
      "response_type": "text",
      "values": [
        {
          "text": "Goodbye."
        }
      ],
      "selection_policy": "sequential"
    },
    {
      "response_type": "end_session"
    }
  ]
}

Injecting custom values into CDR log events

If you are using a log webhook to log call detail record (CDR) events, you can use the cdr_custom_data context variable to add custom data to logged events. You can use this method to record data during a call (for example, to indicate the completion of a specific task).

To log custom CDR data, use the JSON editor to edit the context. Define cdr_custom_data as a child of the context.integrations.voice_telephony object, as in this example:

  "context": {
    "integrations": {
      "voice_telephony": {
        "cdr_custom_data": {
          "key1": "value1",
          "key2": "value2"
        }
      }
    }
  }

The cdr_custom_data object can contain any valid JSON data.

When you generate a CDR report, the custom data is included in the injected_custom_data field, as in this example:

{
  "payload": {
  ...
    "injected_custom_data": {
      "key1": "value1",
      "key2": "value2"
    }
  ...
  }
}

For more information about the structure of the CDR log event payload, see CDR log event reference.

  "context": {
    "integrations": {
      "voice_telephony": {
        "cdr_custom_data": {
          "key1": ""
        }
      }
    }
  }

Access phone integration context variables from your action

If you want to access the phone integration context variables, use the JSON editor to edit the context.

The following example shows how to access the user phone number (the phone number that the call was received from):

  "context": {
    "variables": [
      {
        "value": {
          "expression": "${system_integrations.voice_telephony.private.user_phone_number}.replace('+','')"
        },
        "skill_variable": "user_phone_number"
      }
    ]
  }

For more information, see Phone integration context variables.