AI Agent
The AI Agent scenario block allows you to automate purposeful interactions with customers in both voice and chat scenarios using language models from AI provider integrations. You define any number of requested outcomes and instruct the model how to behave. The AI Agent block uses the language model to conduct a voice or chat dialog with the customer, following your instructions and gathering information for the requested outcomes. Once the block has gathered enough information to reach a desired outcome, the block exits, and scenario variables are populated with data extracted from the dialog.
|
Prerequisites:
| |
How the Block Holds Conversations
The AI Agent block conducts conversations with customers according to the following steps:
- Initialization: When a scenario executes the AI Agent block (dialog block), the AI provider (AI) is used to create the first message for the customer based on the setting Instructions to AI and the list of Requested Outcomes.
- Customer Interaction (Voice or Chat):
- Voice: The AI's message is played to the customer using your default text-to-speech provider. The customer's response is converted to text using the Speech-to-Text integration indicated by the Transcriber setting.
- Chat: The AI's message is sent to the customer as a chat message. The dialog block waits for the customer's text response.
- If the Reply timeout elapsed without a response from the customer, the AI is informed there was "no reply" from the customer.
- Response Evaluation: The customer's response is analyzed by the AI.
- If the customer response contains data related to any of the Requested Outcomes, the AI extracts and formats the data for further use by the dialog block.
- If all the required data has been collected for a Requested Outcome, the AI indicates which outcome the dialog block should follow.
- The AI generates the next message for the customer. The next message is only sent to the customer if the dialog block continues the conversation.
- Exit or continue: The AI Agent block determines whether to continue the conversation or exit the block.
- Outcome Achieved: If all the data for one of the Requested Outcomes has been collected and the AI indicated which outcome to follow, the dialog block exits via the corresponding conditional exit without sending the next message to the customer.
- Continue Dialog: If there is not enough data to satisfy any of the Requested Outcomes, the dialog block returns to the customer interaction step, sending the next message to the customer and awaiting a response.
- Maximum dialog turns: If the customer does not respond after several messages from the AI, or the maximum number of dialog turns have passed, then the dialog block follows the default exit.
Settings
Title Text
The name of the instance of the block in the scenario.
AI Provider
Select the AI provider account to use for this block.
Transcriber
Only for AI Agent blocks used in voice scenarios.
The speech-to-text service that will be used to transcribe the user's statements. Select from among the configured Speech-to-Text (STT) integration accounts.
Custom end of silence time
Only for AI Agent blocks used in voice scenarios.
The duration of silence after the customer's reply that indicates they have finished speaking. When the customer begins speaking, their reply is recorded until there is a pause with duration equal to the Custom end of silence time, at which point their reply is considered finished and is sent to the AI provider for evaluation.
Allow interrupting AI prompt by customer voice
Only for AI Agent blocks used in voice scenarios.
When enabled, the block constantly listens for the customer's voice while an AI message is playing. When a voice is detected during an AI message, the block will:
- Stop playing the AI message
- Record and transcribe the customer's response
- Continue the interaction by sending the customer's response to the AI provider for evaluation
When disabled, the block will begin to listen for the customer's voice only after the entire AI message has been played. Any voice input from the customer before the AI message ends is ignored.
Speech contexts
An optional list of words or phrases likely to be said by the customer. Listed terms are more likely to be transcribed accurately. See the Speech context setting of the Play-Listen block for more information.
Prompt to play while waiting for AI
Only for AI Agent blocks used in voice scenarios.
The selected audio prompt will play while the block waits for the AI provider to return a response. You may find that a musical prompt or sound effect helps customers understand that the system is processing their response.
Instructions to AI
These instructions are given to the AI during every turn of the dialog, allowing you to provide guidance on how the AI should conduct the conversation with the customer. Scenario variables can be used to personalize the instructions for each customer.
Any instructions to the AI should take into account the particular model you have selected with the AI provider. You may find it useful to consult the Prompt Gallery from Vertex AI and the Prompt Engineering Guide from OpenAI for ideas on how to best instruct the AI.
Regardless of the model or the provider you have selected, typical instructions to the AI include the followingː
- Context to the AI about the conversationː "You are chatting with a customer who may want to do X, Y, or Z..."
- Explanations of the goal the AI should try to achieveː "Find out what the customer wants and collect the necessary information..."
- A description of how the AI should conduct itselfː "Be polite, ask short questions, maintain a professional tone... "
- Set limitations on what topics to talk aboutː "only talk about X, Y, or Z. Only collect information necessary to solve X, Y, or Z. The only company you represent is Bright Pattern"
- Include a persona for the AIː "Introduce yourself as Diana, a virtual assistant for Bright Pattern customers..."
Writing effective instructions for the AI will require some experimentation; it may be helpful to test your instructions with conversations that try to take the AI "off script". For example, try to get the AI to discuss a competitor, make promises on behalf of your company, make undesirable statements, ask inappropriate questions, or disobey the instructions.
Include preceding transcript
When disabled (default setting), each request to the AI provider will include only the conversation transcript from the current AI Agent block. If the scenario continues to a different AI Agent, or leaves the current AI Agent and returns (for example, via a Goto block), the conversation starts with an empty transcript. This configuration is typically effective when the particular AI Agent only needs to be aware of the current conversation and does not need additional context.
When enabled, the entire conversation history between the customer and all preceding AI Agent blocks is included whenever an AI Agent block holds a conversation. This provides additional context to the current AI Agent to understand the full state of the conversation from the customer's point of view. However, this option can result in increased cost and response times from the AI provider because of the increased total transcript length.
Requested Outcomes
Defines the outcomes which you would like the AI to pursue with the customer. Each outcome adds a conditional exit to the AI Agent block, allowing you to define different paths for the scenario based on the outcome. For each outcome, you may define any number of associated data points that must be collected for the block to take that exit. The block will only take an exit when the AI chooses that outcome, and all data points for that outcome have been collected.
Each Requested Outcome has the following propertiesː
Name (Requested Outcome)
The name of the outcome and its corresponding conditional exit. The AI considers the name of each Requested Outcome when evaluating the customer response, so it is important to provide names that accurately describe the outcome.
Data (Requested Outcome)
The data component of each Requested Outcome can have any number of elements. Each element must have both a Name and a Value.
- Nameː The name of the data element is considered by the AI when evaluating the customer response, so it is important that it accurately describes the required data.
- Valueː The value is the scenario variable that the scenario block will populate when it takes the relevant conditional exit.
Example
Suppose you want to allow customers to change their address with the AI Agent block, and you need the customer's old address and their new address in order to make the change. To define such an outcome, you could create a Requested Outcome with the following propertiesː
- Nameː "Change Address"
- Dataː
- Nameː "Old Address" - Valueː "old_address"
- Nameː "New Address" - Valueː "new_address"
This will create a conditional exit named "Change Address". If the AI Agent block takes this conditional exit, the scenario variables "old_address" and "new_address" will be populated with the data provided by the customer during the dialog.
Max Dialog Turns
Enter the maximum number of back-and-forth exchanges allowed between the AI and the customer before the block exits with an error.
AI Response Timeout
Enter the maximum time (in seconds) the block will wait for the AI provider to respond after receiving the customer's input. If no response is received before this time, the block will exit with an error.
Customer Reply Timeout
Enter the maximum time (in seconds) the block will wait for the customer to provide a response after the AI has presented a prompt. If no response is received, the block will request a new message from the AI provider, while indicating that the customer did not reply to the previous message. The maximum number of repeated attempts to elicit a response from the customer is defined by the Max Consecutive Reply Timeouts setting.
Max Consecutive Reply Timeouts
The maximum number of consecutive times that the block will wait the full Customer Reply Timeout period and try to elicit a response from the customer. Once the maximum number of attempts has been made, the block will take the default path and indicate an error.
Conditional Exits
The AI Agent block creates conditional exits based on the Requested Outcomes you define. When the AI detects that a specific outcome has been achieved and all the required data for that outcome has been collected, the scenario will follow the corresponding exit path.
Default Exit
The default exit is the path the scenario will follow if the AI Agent block is unable to fulfill the conditions of any of the conditional exits defined by the Requested Outcomes setting. For example, if the customer did not provide the requested information before the maximum number of dialog turns elapsed, or the AI provider returned an error.
The scenario variables aiAgentErrorCode and aiAgentErrorBody, described below, will contain the information needed to determine why the AI Agent block took the default exit.
Variables
The AI Agent block populates several scenario variables to help you understand how the block is operating.
Token Usage Variables
Tokens are the units AI providers use to measure the length of text being processed, with each token typically corresponding to a single word or part of a word. AI providers usually calculate charges based on the number of input tokens (tokens sent to the model) and output tokens (tokens generated by the model) processed during a billing period. The following scenario variables are populated by the AI Agent block during scenario execution:
- aiAgentTotalInTokens contains the cumulative number of tokens sent to the AI provider during scenario execution.
- aiAgentTotalOutTokens contains the cumulative number of tokens received by the AI provider during scenario execution.
Note that these variable values are cumulative during scenario execution (e.g., they are not reset if an AI Agent block is executed multiple times in a single scenario), but you can set them to zero using the Set Variable block.
Final Reply Variable
When the AI Agent block exits, it does not send the final response from the AI provider to the customer. In the case of a conditional exit, the scenario variables specified by the Requested Outcomes setting are populated, and scenario execution continues along the exit without sending the final AI response to the customer. In other cases, a final response may not necessarily exist; for example, if the AI provider returned an error.
If a final reply was provided by the AI provider, it can be retrieved from this variable:
- aiAgentFinalReply contains the most recent final reply from the AI provider.
Error Variables
If the AI Agent block is unable to take a conditional exit, the following variables will be populated with information to help you determine the reason:
- aiAgentErrorCode contains an integer indicating the type of error. See the table below for possible error codes and their meaning.
- aiAgentErrorBody contains the full API response from the AI provider.
The following table indicates the possible values of the aiAgentErrorCode scenario variable, and the meaning of each value.
| Code | Description |
| 1000 | AI provider account is either not specified or has been deleted. |
| 1001 | Speech-to-Text account is either not specified or has been deleted. Applicable only to voice scenarios. |
| 1002 | The AI Agent scenario block is not enabled by your service provider. |
| 1003 | Authentication error from the AI provider account, indicating the API key is missing, invalid or expired. |
| 1004 | AI provider account was unable to execute the API request, indicating a problem with connectivity. |
| 1005 | Reached the maximum number of dialog turns (as configured by Max Dialog Turns), but no outcome was selected or insufficient data was received for the selected outcome. |
| 1006 | Reached the maximum number of consecutive turns without receiving a customer reply (as configured by Max consecutive timeouts). |
| 1007 | The AI provider did not provide any response or results during the most recent turn. |
| 1008 | The transcriber was not able to recognize the customer response. Applicable only to voice scenarios. |
| 2000 | Other error from the AI provider. The entire API response payload is available in the aiAgentErrorBody variable and can be useful for understanding the error reason.
|