Documentation Index
Fetch the complete documentation index at: https://docs.cekura.ai/llms.txt
Use this file to discover all available pages before exploring further.
What are Structured Tests?
Structured Tests is a specialized evaluator type that allows you to create dynamic, rule-based test scenarios. Unlike traditional evaluators that follow a linear script, Structured Tests evaluators can adapt their behavior based on what the agent says during the conversation. This makes them ideal for testing complex workflows where the conversation can take multiple paths depending on the agent’s responses.When to Use Structured Tests
Use Structured Tests evaluators when you need to:- Test branching conversations: Handle different conversation paths based on agent responses
- Simulate decision trees: Test scenarios where user responses depend on what the agent asks
- Create adaptive evaluators: Build evaluators that can handle unexpected agent responses
- Test IVR (Interactive Voice Response) flows: Simulate interactive voice response systems with menu options
- Validate error handling: Test how agents handle various user inputs and edge cases
Example Use Case
Structure
A Structured Tests evaluator consists of two main components:1. Role
Therole defines who the testing agent is pretending to be.
2. Conditions
Theconditions array defines the rules and actions for the evaluator. Each condition specifies:
- When a certain situation occurs (the condition)
- What the testing agent should do (the action)
Condition Fields
Each condition object has the following fields:id: 0. Each ID must be unique — duplicate IDs are rejected.- When
idis 0, the condition must be the string"FIRST_MESSAGE"(represents the first message) - When
typeisaction_followup, the condition should be an integer referencing the previous condition ID - For standard conditions, the condition is a string describing the trigger
fixed_message field:- When
fixed_messageisfalse: Contains instructions for the testing agent to interpret - When
fixed_messageistrue: Contains the exact message to send word-for-word
FIRST_MESSAGE condition (id: 0) may have an empty action when the main agent speaks first.standard: Normal condition triggered by the conversation contextaction_followup: Fires on the next turn after the referenced condition — the testing agent sends the referenced condition’s message, waits for the main agent to reply, then sends this message
true or false on every condition — omitting it returns a validation error.true: Theactionfield contains the exact message to send word-for-wordfalse: Theactionfield contains instructions for the testing agent to interpret naturally
fixed_message: true.Validation Rules
The backend enforces the following constraints on all Structured Tests evaluators. Violations return a validation error.1. FIRST_MESSAGE required
1. FIRST_MESSAGE required
conditions array must satisfy all three requirements:conditionmust equal"FIRST_MESSAGE"(not empty, not any other string)idmust be0fixed_messagemust betrue
2. Non-empty actions
2. Non-empty actions
action field cannot be empty or whitespace-only. This rule applies to all conditions except FIRST_MESSAGE (id: 0), which may have an empty action when the main agent speaks first.3. type is required
3. type is required
type field. There is no default — omitting type returns a validation error.4. fixed_message is required
4. fixed_message is required
fixed_message field set to true or false. Omitting it returns a validation error.5. Unique condition IDs
5. Unique condition IDs
id. Duplicate IDs are rejected.6. Language required
6. Language required
scenario_language. Set it by assigning a personality with a language configured. If you provide a personality, the language is inferred automatically; if no personality is set, scenario_language must be explicitly provided.This also applies when changing an existing evaluator’s type to Structured Tests.First Message (id: 0)
The condition withid: 0 is special—it represents the first message the testing agent sends to start the conversation. Three rules apply to this condition:
conditionmust be the string"FIRST_MESSAGE"fixed_messagemust betrueactionmay be empty if the main agent speaks first
action to an empty string:
Condition Types
Standard Conditions
Standard conditions are evaluated based on the conversation context. They trigger when the specified condition is met.Action Followup Conditions
Anaction_followup fires on the next turn after the condition it references. It does not fire immediately. The sequence is:
- Testing agent sends the message from condition X
- Main agent receives it and replies
action_followup(withcondition: X) fires — testing agent sends this message
action_followup has two key uses:
- Multi-part responses: send several messages across consecutive turns after a single trigger
- Scripted sequences: chain followups to deliver an exact sequence of messages from the testing agent, turn by turn, without needing any conditions to match — the sequence fires automatically regardless of what the main agent says
condition field must be an integer referencing the id of the previous condition.
action_followup does not fire immediately after condition X — it fires on the next turn, after the main agent has replied to condition X. The condition field holds the id (integer) of the condition whose turn triggers this followup.Action vs Fixed Message
Theaction field can be used in two ways, controlled by the fixed_message boolean:
Using action with fixed_message: false
When fixed_message is false, the action field provides instructions that the testing agent interprets to generate a natural response.
Using action with fixed_message: true
When fixed_message is true, the action field contains the exact text that the testing agent will say, word-for-word.
When to Use Each
Use fixed_message: false
- Natural, varied responses
- Testing agent adaptability
- When exact wording doesn’t matter
Use fixed_message: true
- Exact phrases required
- Testing specific keywords
- Compliance testing
- Reproducible test cases
- First message (id: 0) — always required
- Actions containing XML tags
fixed_message is required on every condition and must be explicitly set to true or false. It determines how action is interpreted: as instructions (false) or as the exact message to send (true).Supported Tags
Conditional Actions supports a variety of special tags in theaction field to control conversation flow and behavior.
Communication Tags
<ivr> - Interactive Voice Response
<ivr> - Interactive Voice Response
<ivr>. When a scenario contains the <ivr> tag, any DTMF digits pressed by the main agent (the agent being tested) are captured and appear in the transcript. This lets you write conditions that detect the exact digit pressed — for example, "The main agent pressed 1" — rather than relying on speech detection alone. Both <ivr> and <dtmf> can appear in any condition, not just id: 0.text: The IVR message to play (required)
<voicemail> - Voicemail Greeting
<voicemail> - Voicemail Greeting
text: Voicemail greeting message (optional) - if omitted, only plays beep; if provided, cannot be empty
<endcall> - End Call
<endcall> - End Call
Speech Control Tags
<silence> vs <hold> — both add pauses, but behave differently:<silence> | <hold> | |
|---|---|---|
| Interruptible | ✅ Yes — main agent can interrupt | ❌ No — testing agent won’t be interrupted |
| After interruption | Condition matching restarts after main agent stops speaking | N/A |
| Background noise | Continues to play during the pause | Does not play during the pause |
<silence> when you want the conversation to feel natural and allow the main agent to jump in. Use <hold> when you need a guaranteed delay before the next message.<silence> - Add Pauses (interruptible)
<silence> - Add Pauses (interruptible)
time: Duration of silence (required) - format: “Xs” where X is a number (e.g., “1s”, “2.5s”)
<hold> - Hold Between Messages (not interruptible)
<hold> - Hold Between Messages (not interruptible)
time: Duration to wait before next message (required) - format: “Xs” where X is a number (e.g., “5s”, ”10s”)
<spell> - Spell Out Text
<spell> - Spell Out Text
<speed> - Control Speech Speed
<speed> - Control Speech Speed
ratio: Speed multiplier (required) - Valid range: 0.8 to 1.2
Examples:"1.2"for 20% faster,"0.8"for 20% slower
<volume> - Control Volume
<volume> - Control Volume
ratio: Volume multiplier (required) - Valid range: 0 to 2
Examples:"1.5"for 50% louder,"0.5"for 50% quieter
Interaction Tags
<dtmf> - Send DTMF Tones
<dtmf> - Send DTMF Tones
digits: DTMF digits to send (required) - e.g., “123”, “456#”, “*9”
<send_sms> - Send SMS
<send_sms> - Send SMS
text: SMS message content to send (required)
<interruption> - Interrupt After Timeout
<interruption> - Interrupt After Timeout
time: Duration to wait before interrupting (required) - format: “Xs” where X is a number (e.g., “3s”, “5.5s”)
Environmental Tags
<background_noise> - Background Sounds
<background_noise> - Background Sounds
sound: Sound name (required) - see supported sounds belowvolume: Volume level (optional) - default applies if not specified
| Sound name | Description |
|---|---|
office-ambience | Office environment |
coffee-shop | Coffee shop chatter |
kitchen-noise | Kitchen sounds |
home-chatter | Home background voices |
vacuum-cleaner | Vacuum cleaner |
dog-barking | Dog barking |
baby-crying | Baby crying |
keyboard-typing | Keyboard typing |
coughing | Coughing sounds |
background-printer | Printer noise |
quiet-room | Near-silent room |
air-conditioner | AC hum |
construction-site | Construction noise |
busy-street | Street traffic |
airport-boarding | Airport gate |
inside-car | Moving car interior |
inside-train | Moving train interior |
public-park | Park ambience |
rain-thunder | Rain and thunder |
windy-day | Wind |
restaurant | Restaurant ambience |
shopping-mall | Shopping mall |
stadium-crowd | Crowd noise |
standard-hiss | White noise hiss |
static-radio | Radio static |
fan-buzz | Fan buzz |
ship-humming | Low ship hum |
two-people-talking | Background conversation |
train-station | Train station hall |
holding-on-song | Hold music |
<background_noise> tag in evaluator actions. To use a custom audio file URL as background noise (e.g., https://your-domain.com/noise.mp3), use the Custom URL option in the Background Noise dropdown when creating or editing a personality — no tag required.<noise> - Play Sound Effects
<noise> - Play Sound Effects
sound: Sound name (required) - supported:office,beep,cough1,cough2volume: Volume level (optional)time: Duration in ms (optional)
<network_simulation> - Simulate Network Issues
<network_simulation> - Simulate Network Issues
packet_loss: Packet loss percentage (required) - e.g.,"5"= 5% loss
Complete Examples
Example 1: Appointment Cancellation
Example 2: IVR Navigation
- Inbound (main agent IS the IVR): Use
<dtmf>to navigate the IVR menu played by the main agent.<dtmf>can appear in any condition. - Outbound (testing agent simulates an external IVR): Use
<ivr text="..." />to play the IVR menu.<ivr>can appear in any condition. When the scenario contains<ivr>, DTMF digits pressed by the main agent appear in the transcript — write your conditions using the specific digit (e.g.,"The main agent pressed 1").
Example 3: Multi-Part Response with Followup
- Testing agent sends condition 0: “Hi, I need help with my order”
- Main agent replies
- Action followup (condition: 0) fires — testing agent: “Also mention you haven’t received a confirmation email”
- Main agent replies
- Action followup (condition: 1) fires — testing agent: “And ask if you can get expedited shipping”
- Main agent replies — condition 3 (standard) matches and fires
Example 4: Using Advanced Tags
Best Practices
Clear Conditions
- ✅ “The agent asks for your email”
- ❌ “Email”
Specific Actions
- ✅ “Provide your email address”
- ❌ “Answer the question”
Logical Flow
id: 0 (FIRST_MESSAGE), then follow natural conversation progression.Common Patterns
Pattern 1: Conditional Branching
Handle different paths based on agent response:Pattern 2: Information Gathering
Progressively provide information as requested:Pattern 3: Scripted Sequence with action_followup
Chainaction_followup conditions to deliver an exact sequence of messages across turns, with no conditions to match — each followup fires automatically on the next turn after the main agent replies, regardless of what the main agent says:
Tips
Troubleshooting
Validation error: first condition must have condition='FIRST_MESSAGE'
Validation error: first condition must have condition='FIRST_MESSAGE'
conditions array has all three:condition was left empty (""); it must now be the exact string "FIRST_MESSAGE".Validation error: type is required
Validation error: type is required
type field.Solution: Add type explicitly to every condition. There is no default:Validation error: fixed_message is required
Validation error: fixed_message is required
fixed_message field.Solution: Add fixed_message explicitly to every condition set to true or false:Validation error: duplicate condition ID
Validation error: duplicate condition ID
id.Solution: Ensure every condition has a unique integer id. Use sequential numbering (0, 1, 2, …) to avoid collisions.Validation error: scenario_language is required
Validation error: scenario_language is required
scenario_language explicitly in the request.Condition Not Triggering
Condition Not Triggering
- Make the condition more specific
- Check if a previous condition is matching instead
- Verify the condition describes what the agent says, not what the evaluator should do
action_followup firing unexpectedly or not at all
action_followup firing unexpectedly or not at all
action_followup fires at the wrong time or doesn’t fire.Explanation: action_followup fires on the next turn after the referenced condition — after the testing agent sends condition X and the main agent replies. It does not fire in the same turn as condition X.Solutions:- Ensure
conditionpoints to the correct ID of the preceding condition - If chaining multiple followups, each one references the ID of the previous followup (not the original condition)
Tags Not Working
Tags Not Working
First Message Not Sending
First Message Not Sending
- Ensure you have a condition with
id: 0andcondition: "FIRST_MESSAGE" - Verify
fixed_message: trueis set on the first condition - Check that the role is defined
- If the agent speaks first, set
actionto""(empty string is allowed for FIRST_MESSAGE only)