​
Python Code Metrics

Python Code Metrics allow you to write custom evaluation logic in Python to evaluate your AI agent’s performance. This gives you complete control over the evaluation process and enables complex analysis that goes beyond simple prompt-based metrics.

​
Overview

Custom code metrics are executed in a secure Python environment with access to call data including transcripts, metadata, and dynamic variables. Your code must set specific output variables to provide the evaluation result and explanation.

​
Available Data Variables

When writing your custom code, you have access to the following data variables:

​
Required Output Variables

Your Python code must set these two variables:
  • _result - The evaluation outcome (can be boolean, numeric, string, etc.)
  • _explanation - A string explaining the reasoning behind the result

​
Example Code

Here’s a simple example that checks if the agent mentioned a specific product: 
# Check if the agent mentioned "Premium Plan" in the conversation
transcript = data["transcript"].lower()
if "premium plan" in transcript:
    _result = True
    _explanation = "Agent successfully mentioned the Premium Plan during the conversation"
else:
    _result = False
    _explanation = "Agent did not mention the Premium Plan in the conversation"

​
Using Metric Results

You can access the results of other metrics that were evaluated for the same call directly by metric name using data[<metric_name>]. This allows you to create complex evaluations that depend on the outcomes of other metrics. Example usage: 
# Access metric results directly by name
customer_satisfaction = data["Customer Satisfaction"]
response_time = data["Response Time"]
product_knowledge = data["Product Knowledge"]

# Each metric result contains the evaluation outcome
if customer_satisfaction and response_time < 60:
    _result = "Excellent"
    _explanation = "Customer was satisfied and response time was fast"

​
Advanced Example

Here’s a more complex example that analyzes sentiment and response time: 
import re
from datetime import datetime

# Get transcript data
transcript = data["transcript"]
call_duration = data["call_duration"]

# Analyze agent responses
agent_responses = []
lines = transcript.split('\n')

for line in lines:
    if line.strip().startswith('Agent:'):
        response = line.replace('Agent:', '').strip()
        agent_responses.append(response)

# Calculate average response length
if agent_responses:
    avg_response_length = sum(len(response) for response in agent_responses) / len(agent_responses)

    # Check if responses are detailed enough (more than 50 characters average)
    if avg_response_length > 50:
        _result = True
        _explanation = f"Agent provided detailed responses with average length of {avg_response_length:.1f} characters"
    else:
        _result = False
        _explanation = f"Agent responses were too brief with average length of {avg_response_length:.1f} characters"
else:
    _result = False
    _explanation = "No agent responses found in transcript"

​
Example Using Multiple Data Sources

Here’s an example that combines multiple metric results with call metadata and tags: 
# Access metric results directly by name
try:
    satisfaction = data["Customer Satisfaction"]
    response_time = data["Response Time"]

    # Access additional call data
    call_duration = data["call_duration"]
    call_ended_reason = data["call_ended_reason"]
    tags = data["tags"]

    # Check if this was a priority call based on tags
    is_priority = "priority" in tags or "vip" in tags

    # Evaluate based on multiple factors
    if call_ended_reason == "hangup" and satisfaction and response_time < 60:
        if is_priority:
            _result = "Excellent"
            _explanation = f"Priority customer was satisfied ({satisfaction}) with fast response time ({response_time}s) and completed the call normally"
        else:
            _result = "Good"
            _explanation = f"Customer was satisfied ({satisfaction}) with fast response time ({response_time}s) and completed the call normally"
    elif call_ended_reason in ["timeout", "error"]:
        _result = "Poor"
        _explanation = f"Call ended unexpectedly due to {call_ended_reason}, indicating technical issues"
    else:
        _result = "Needs Improvement"
        _explanation = f"Call performance needs improvement - satisfaction: {satisfaction}, response time: {response_time}s, ended reason: {call_ended_reason}"

except KeyError as e:
    _result = "Incomplete"
    _explanation = f"Required data not found: {str(e)}"