Skip to main content
These are standard metrics applicable across domains defined by Cekura. Below is a list of supported pre-defined metrics.
  • AI Interrupting User: Tells if the AI (main agent) interrupted the user (testing agent) during the interaction. NOTE: This only works on stereo recordings.
  • Average Latency (in ms): Tells the average response latency for the AI agent in milliseconds.
  • Average Pitch (in Hz): Returns the average pitch of the AI Agent (Main agent) during the call in Hertz.
  • Instruction Following Metric: Checks critical deviations from the expected workflows and tells where it deviated.
  • CSAT: Gives a score for customer satisfaction and tells the reason for dissatisfaction.
  • Not Early Termination: Checks if the call was ended early by the user, indicating poor user experience or unresolved issues. This metric is True if…
  • Sentiment: Determines whether the human’s overall sentiment towards the AI agent during the conversation was “happy”…
  • Signal to Noise Ratio (SNR): Signal to Noise Ratio when the AI agent (Main agent) is speaking. Compares the noise level of Main Agent against Testing Agent.
  • Transcription Accuracy: Evaluates the accuracy of speech-to-text transcription by comparing the transcript against reference transcriptions. Returns a score from 1-5 where higher scores indicate better transcription accuracy. For simulations/runs, compares the Cekura transcript against the provider’s transcript using Word Error Rate (WER), with the final score based on weighted errors involving significant words (proper nouns, common nouns, and numbers at full weight; verbs at half weight). For call logs, transcribes the testing agent’s audio channel using Gemini and ElevenLabs, then uses an LLM to compare against the candidate transcript.
  • Talk Ratio: The ratio of time duration the AI Agent (Main agent) is speaking compared to the duration the User (Test Agent) is speaking.
  • User Interrupting AI: Tells if the user (testing agent) interrupted the AI (main agent) during the interaction. NOTE: This only works on stereo recordings.
  • Voice Quality Index: Evaluates the overall voice quality of the AI agent based on three key factors: clarity (how clear and understandable the voice is), tone (appropriateness of tone for the context), and appropriateness (whether the speech fits the context and intent). Returns a score from 0-5 where higher scores indicate better voice quality. NOTE: This metric requires audio recordings for analysis.
  • Response Consistency: Checks if the responses of your AI Agent are consistent across the duration of a call. This metric detects two specific cases:
    • When a user provides information (like their name) and the AI agent repeats it back incorrectly
    • When the AI agent provides contradictory information during the same call (e.g., saying X at the start and Y at the end which contradicts the earlier statement)
  • Words Per Minute (WPM): Monitors speech speed of AI Agent (Main Agent) to ensure natural and understandable delivery.