Create Evaluator from Transcript
Create a test scenario from a call transcript
Authorizations
API Key Authentication. It should be included in the header of each request.
Body
JSON representation of the conversation transcript
ID of the personality to use in the simulation
ID of the agent to use in the simulation
Assistant ID to use in the simulation
Name for the simulated scenario
Prompt describing the expected outcome of the scenario
Response
Name of the scenario
80Scenario Instructions - format depends on scenario_type:
For scenario_type="instruction":
Plain text describing the caller's behavior.
Example: "You are a customer calling to inquire about your order status. Be polite and provide order number 12345 when asked."
For scenario_type="conditional_actions":
Use the structured conditional_actions field instead. If you must send via this field, pass a JSON-formatted string with role and conditions:
{
"role": "A friendly customer calling about an appointment",
"conditions": [
{
"id": 0,
"condition": "FIRST_MESSAGE",
"action": "Hi, I'd like to check on my upcoming appointment <silence time=\"1.5s\" />",
"type": "standard",
"fixed_message": true
},
{
"id": 1,
"condition": "The agent asks for your name",
"action": "My name is Sarah Johnson",
"type": "standard",
"fixed_message": true
},
{
"id": 2,
"condition": "The agent asks for your date of birth",
"action": "January first, nineteen ninety",
"type": "standard",
"fixed_message": true
},
{
"id": 3,
"condition": "The agent confirms your identity and provides appointment details",
"action": "Thank you, that's all I needed <silence time=\"1.0s\" />",
"type": "standard",
"fixed_message": true
},
{
"id": 4,
"condition": 3,
"action": "<endcall />",
"type": "action_followup",
"fixed_message": true
}
]
}Structure:
role: Personality/role descriptionconditions: Array with id (0,1,2...), condition (FIRST_MESSAGE | a natural-language third-person description of the main agent's observable action, e.g."The agent asks for your date of birth"— matched semantically against the conversation context | integer reference to another condition's id), action (text with SSML<silence time="1.5s" />(interruptible) or<hold time="2s" />(not interruptible) and markers[laughter]or<endcall />), type (standard | action_followup), fixed_message (boolean)
Foreign key reference to the AI agent that owns this scenario
Unique identifier for the AI agent
Example: 123
Foreign key reference to the personality used in this scenario
Name of the personality used in this scenario
Example: "Normal Male"
Name of the agent associated with this scenario
Example: "Customer Support Agent"
Expected outcome prompt for the scenario
Example: "The user should be able to complete the order"
Language code for the scenario (ISO language code)
af- Afrikaansar- Arabicbn- Bengalibg- Bulgarianzh- Chinese Simplifiedcs- Czechda- Danishnl- Dutchen- Englishet- Estonianfi- Finnishfr- Frenchde- Germanel- Greekgu- Gujaratihi- Hindihe- Hebrewhu- Hungarianid- Indonesianit- Italianja- Japanesekn- Kannadako- Koreanms- Malayml- Malayalammr- Marathimulti- Multilingualno- Norwegianpl- Polishpa- Punjabipt- Portuguesero- Romanianru- Russiansk- Slovakes- Spanishsv- Swedishth- Thaitr- Turkishtl- Tagalogta- Tamilte- Teluguuk- Ukrainianvi- Vietnamese
af, ar, bn, bg, zh, cs, da, nl, en, et, fi, fr, de, el, gu, hi, he, hu, id, it, ja, kn, ko, ms, ml, mr, multi, no, pl, pa, pt, ro, ru, sk, es, sv, th, tr, tl, ta, te, uk, vi, null Type of scenario:
instruction: Standard instruction-based scenario where instructions field contains plain textconditional_actions: Conditional action-based scenario where instructions field contains JSONreal_world_smart: Real world smart scenarioreal_world_fixed: Real world fixed scenario
instruction- Instructionreal_world_smart- Real World Smartreal_world_fixed- Real World Fixedconditional_actions- Conditional Actions
instruction, real_world_smart, real_world_fixed, conditional_actions List of tags associated with this scenario for categorization
Example: ["tag1", "tag2", "tag3"]
(Deprecated) Foreign key reference to the phone number. Use phone_number field instead. This field is maintained for backward compatibility and will be automatically synced with phone_number.
Details of the phone number used for inbound calls
List of metrics associated with this scenario for evaluation
List of metric names associated with this scenario
Example: ["Metric 1", "Metric 2", "Metric 3"]
The initial message that the AI agent will send when the scenario starts
List of tool IDs that are available for use in this scenario
Example: ["TOOL_DTML", "TOOL_END_CALL"]
Test profile ID
Example: 123
Details of the test profile associated with this scenario Example:
{
"id": "<integer>",
"agent": "<integer>",
"name": "<string>",
"information": {
"user_name": "John Doe",
"user_email": "john.doe@example.com",
}
}Generated values for the agent's dynamic variables for this scenario. Keys are variable names; values are the generated values.
Timestamp when the scenario was created
Timestamp when the scenario was last updated
ID of the user who created this scenario
ID of the user who last updated this scenario
Dot-separated path of the folder to assign this evaluator to (e.g. "Sales.Inbound"). Folders must be created before use — they are not auto-created. Use scenarios_create_folder_create to create each level before assigning evaluators. Example: to use "Sales.Inbound", first create "Sales", then create "Inbound" inside it.