• Checks critical deviations from the agent instructions and tells where it deviated.
  • This metric can be found in the list of pre-defined metrics.
  • Issues are categorized into:
    • Type of Issue: The nature or category of the problem, such as technical errors, user errors, or system limitations.
    • Scenario: The specific context or situation in which the issue occurred, helping to understand the conditions that led to the deviation.
  • Issues are categorized based on priority:
    • High Priority: Issues that need immediate attention and can significantly impact the workflow. Alerts can be set up for these issues to ensure prompt action.
    • Medium Priority: Issues that are important but do not require immediate action.
    • Low Priority: Minor issues that have minimal impact on the workflow.
    • Not a Bug: Identified issues that are not considered bugs and do not require changes.
  • Example: Suppose an AI customer support agent for a SaaS provider is expected to transfer the call to human in case the customer reports a technical challenge. Now the technical challenge can be login credentials not working. In this scenario, a call transfer is important and failing to do so is a high priority bug. However, if the technical challenge is internet is not working, then a call transfer is unnecessary. You can mark that scenario as “Not a Bug” and the next time your AI voice agent detects this edge case, you won’t be bothered.
  • The results of this metric will be listed in its own section in a call as seen below.

In this example, the priority can be seen in the dropdown. The type of issue is The AI Agent did not confirm any time in line with steps (3) and (4) and the Scenario of issue is The AI Agent failed to confirm the time deviating from expected scheduling protocol