Structured Tests

What are Structured Tests?

Structured Tests is a specialized evaluator type that allows you to create dynamic, rule-based test scenarios. Unlike traditional evaluators that follow a linear script, Structured Tests evaluators can adapt their behavior based on what the agent says during the conversation. This makes them ideal for testing complex workflows where the conversation can take multiple paths depending on the agent’s responses.

When to Use Structured Tests

Use Structured Tests evaluators when you need to:

Test branching conversations: Handle different conversation paths based on agent responses
Simulate decision trees: Test scenarios where user responses depend on what the agent asks
Create adaptive evaluators: Build evaluators that can handle unexpected agent responses
Test IVR (Interactive Voice Response) flows: Simulate interactive voice response systems with menu options
Validate error handling: Test how agents handle various user inputs and edge cases

Example Use Case

Testing an appointment booking agent where the flow changes based on whether the agent offers available slots, asks for rescheduling, or cancels the appointment entirely.

Structure

A Structured Tests evaluator consists of two main components:

1. Role

The role defines who the testing agent is pretending to be.

{
  "role": "You are a customer calling to cancel an appointment"
}

Keep the role concise and clear. It sets the context for the entire conversation.

2. Conditions

The conditions array defines the rules and actions for the evaluator. Each condition specifies:

When a certain situation occurs (the condition)
What the testing agent should do (the action)

{
  "role": "You are a customer calling to cancel an appointment",
  "conditions": [
    {
      "id": 0,
      "condition": "FIRST_MESSAGE",
      "action": "Hi, I need to cancel my appointment",
      "type": "standard",
      "fixed_message": true
    },
    {
      "id": 1,
      "condition": "The agent asks for your name",
      "action": "Provide your name as John Smith",
      "type": "standard",
      "fixed_message": false
    },
    {
      "id": 2,
      "condition": "The agent asks for a reason",
      "action": "Say you have a scheduling conflict",
      "type": "standard",
      "fixed_message": false
    }
  ]
}

Condition Fields

Each condition object has the following fields:

integer

required

Unique identifier for the condition. The first condition must use id: 0. Each ID must be unique — duplicate IDs are rejected.

string | integer

required

The trigger condition that determines when this action should be taken.Special cases:

When id is 0, the condition must be the string "FIRST_MESSAGE" (represents the first message)
When type is action_followup, the condition should be an integer referencing the previous condition ID
For standard conditions, the condition is a string describing the trigger

string

required

What the testing agent should say or do when the condition is met. Cannot be empty or whitespace-only. The interpretation depends on the fixed_message field:

When fixed_message is false: Contains instructions for the testing agent to interpret
When fixed_message is true: Contains the exact message to send word-for-word

Exception: The FIRST_MESSAGE condition (id: 0) may have an empty action when the main agent speaks first.

string

required

The condition type. Must be explicitly provided — there is no default. Options:

standard: Normal condition triggered by the conversation context
action_followup: Fires on the next turn after the referenced condition — the testing agent sends the referenced condition’s message, waits for the main agent to reply, then sends this message

boolean

required

Must be explicitly set to true or false on every condition — omitting it returns a validation error.

true: The action field contains the exact message to send word-for-word
false: The action field contains instructions for the testing agent to interpret naturally

Note: The FIRST_MESSAGE condition (id: 0) must have fixed_message: true.

Validation Rules

The backend enforces the following constraints on all Structured Tests evaluators. Violations return a validation error.

1. FIRST_MESSAGE required

The first condition in the conditions array must satisfy all three requirements:

condition must equal "FIRST_MESSAGE" (not empty, not any other string)
id must be 0
fixed_message must be true

// ✅ Valid
{ "id": 0, "condition": "FIRST_MESSAGE", "action": "Hello", "type": "standard", "fixed_message": true }

// ❌ Invalid — condition is empty
{ "id": 0, "condition": "", "action": "Hello", "type": "standard", "fixed_message": true }

// ❌ Invalid — fixed_message is missing
{ "id": 0, "condition": "FIRST_MESSAGE", "action": "Hello", "type": "standard" }

2. Non-empty actions

The action field cannot be empty or whitespace-only. This rule applies to all conditions except FIRST_MESSAGE (id: 0), which may have an empty action when the main agent speaks first.

// ✅ Valid — FIRST_MESSAGE may have empty action
{ "id": 0, "condition": "FIRST_MESSAGE", "action": "", "type": "standard", "fixed_message": true }

// ❌ Invalid — non-FIRST_MESSAGE condition has empty action
{ "id": 1, "condition": "The agent greets you", "action": "", "type": "standard", "fixed_message": false }

3. type is required

Every condition must include a type field. There is no default — omitting type returns a validation error.

// ✅ Valid
{ "id": 1, "condition": "Agent asks for name", "action": "Provide name", "type": "standard", "fixed_message": false }

// ❌ Invalid — type is missing
{ "id": 1, "condition": "Agent asks for name", "action": "Provide name", "fixed_message": false }

4. fixed_message is required

Every condition must include a fixed_message field set to true or false. Omitting it returns a validation error.

// ✅ Valid
{ "id": 1, "condition": "Agent asks for name", "action": "Provide name", "type": "standard", "fixed_message": false }

// ❌ Invalid — fixed_message is missing
{ "id": 1, "condition": "Agent asks for name", "action": "Provide name", "type": "standard" }

5. Unique condition IDs

Every condition must have a unique id. Duplicate IDs are rejected.

// ❌ Invalid — id 1 appears twice
[
  { "id": 0, "condition": "FIRST_MESSAGE", "action": "Hello", "type": "standard", "fixed_message": true },
  { "id": 1, "condition": "Agent asks for name", "action": "Provide name", "type": "standard", "fixed_message": false },
  { "id": 1, "condition": "Agent asks for email", "action": "Provide email", "type": "standard", "fixed_message": false }
]

6. Language required

Structured Tests evaluators require a scenario_language. Set it by assigning a personality with a language configured. If you provide a personality, the language is inferred automatically; if no personality is set, scenario_language must be explicitly provided.This also applies when changing an existing evaluator’s type to Structured Tests.

First Message (id: 0)

The condition with id: 0 is special—it represents the first message the testing agent sends to start the conversation. Three rules apply to this condition:

condition must be the string "FIRST_MESSAGE"
fixed_message must be true
action may be empty if the main agent speaks first

{
  "id": 0,
  "condition": "FIRST_MESSAGE",
  "action": "Hi, I need to cancel my appointment",
  "type": "standard",
  "fixed_message": true
}

When the main agent speaks first, set action to an empty string:

{
  "id": 0,
  "condition": "FIRST_MESSAGE",
  "action": "",
  "type": "standard",
  "fixed_message": true
}

The condition field for the first message must be "FIRST_MESSAGE" exactly. An empty string or any other value will fail validation.

Condition Types

Standard Conditions

Standard conditions are evaluated based on the conversation context. They trigger when the specified condition is met.

{
  "id": 1,
  "type": "standard",
  "condition": "The agent asks for your account number",
  "action": "Provide your account number",
  "fixed_message": false
}

Action Followup Conditions

An action_followup fires on the next turn after the condition it references. It does not fire immediately. The sequence is:

Testing agent sends the message from condition X
Main agent receives it and replies
action_followup (with condition: X) fires — testing agent sends this message

The main agent’s reply is received but does not affect whether the followup fires. Because of this, action_followup has two key uses:

Multi-part responses: send several messages across consecutive turns after a single trigger
Scripted sequences: chain followups to deliver an exact sequence of messages from the testing agent, turn by turn, without needing any conditions to match — the sequence fires automatically regardless of what the main agent says

The condition field must be an integer referencing the id of the previous condition.

{
  "id": 2,
  "type": "action_followup",
  "condition": 1,
  "action": "Also mention that you need a confirmation email",
  "fixed_message": false
}

action_followup does not fire immediately after condition X — it fires on the next turn, after the main agent has replied to condition X. The condition field holds the id (integer) of the condition whose turn triggers this followup.

Action vs Fixed Message

The action field can be used in two ways, controlled by the fixed_message boolean:

Using `action` with `fixed_message: false`

When fixed_message is false, the action field provides instructions that the testing agent interprets to generate a natural response.

{
  "id": 1,
  "type": "standard",
  "condition": "The agent asks for your name",
  "action": "Provide your name as John Smith",
  "fixed_message": false
}

The testing agent might say: “My name is John Smith” or “Sure, it’s John Smith” or “John Smith”

Using `action` with `fixed_message: true`

When fixed_message is true, the action field contains the exact text that the testing agent will say, word-for-word.

{
  "id": 1,
  "type": "standard",
  "condition": "The agent asks for your name",
  "action": "My name is John Smith",
  "fixed_message": true
}

The testing agent will say exactly: “My name is John Smith”

When to Use Each

Use fixed_message: false

Natural, varied responses
Testing agent adaptability
When exact wording doesn’t matter

Use fixed_message: true

Exact phrases required
Testing specific keywords
Compliance testing
Reproducible test cases
First message (id: 0) — always required
Actions containing XML tags

fixed_message is required on every condition and must be explicitly set to true or false. It determines how action is interpreted: as instructions (false) or as the exact message to send (true).

Supported Tags

Conditional Actions supports a variety of special tags in the action field to control conversation flow and behavior.

Important: Tags are only supported when fixed_message is set to true. They will not work with fixed_message: false (instruction-based actions).

Tags can be combined with text and other sibling tags. They run from left to right; do not nest tags inside one another, except inside a regional <voice ...>...</voice> block. <ivr> and <voicemail> remain whole-action tags.

Communication Tags

<ivr> - Interactive Voice Response

Mark a message as an IVR (Interactive Voice Response) message that cannot be interrupted.

{
  "id": 1,
  "type": "standard",
  "condition": "The call is answered",
  "action": "<ivr text=\"Press 1 for sales, press 2 for support\" />",
  "fixed_message": true
}

The agent cannot interrupt while this message plays.

<ivr> must be the entire action — it cannot be combined with other text or tags in the same action string.

DTMF digits appear in the transcript when using <ivr>. When a scenario contains the <ivr> tag, any DTMF digits pressed by the main agent (the agent being tested) are captured and appear in the transcript. This lets you write conditions that detect the exact digit pressed — for example, "The main agent pressed 1" — rather than relying on speech detection alone. Both <ivr> and <dtmf> can appear in any condition, not just id: 0.

Attributes:

text: The IVR message to play (required)

<hold> and <audio> tags are not supported inside <ivr text="..."> and are rejected at save time. To play a sequence of audio clips and pauses without interruption, use <ignore_interruptions> instead.

<ignore_interruptions> - Protected Playback Span

Wrap a span of the action so everything inside plays to completion — nothing the main agent says can interrupt it. Content is unrestricted: spoken text, <audio> clips, <hold>/<silence> pauses, or any mix. The typical use is a multi-part IVR menu of clips and pauses.

{
  "id": 1,
  "type": "action_followup",
  "condition": 0,
  "action": "<ignore_interruptions> <audio id=\"2eeb6544\"/> <hold time=\"5s\" /> <audio id=\"f0c5cdfe\"/> <hold time=\"5s\" /> <audio id=\"1bfc2ad2\"/> </ignore_interruptions> <endcall />",
  "fixed_message": true
}

Unlike <ivr>, this is a block tag scoped to a span: content goes between the opening and closing tags, it can appear more than once per action, and normal text or tags can come before and after it.What the main agent’s speech does during the span:

It does not interrupt playback or abort the remaining clips.
It does not trigger a reply from the testing agent — during the span or after it ends.
It is still transcribed and included in evaluation, so metrics can judge what the agent said over the menu.

Nested spans are not supported — use sequential spans instead. Attributes are not supported; content always goes between the tags.

<voicemail> - Voicemail Greeting

Play a voicemail greeting with beep.

{
  "action": "<voicemail text=\"Hi, you've reached John. Leave a message.\" />"
}

Or just a beep without greeting:

{
  "action": "<voicemail />"
}

<voicemail> must be the entire action — it cannot be combined with other text or tags in the same action string.

Attributes:

text: Voicemail greeting message (optional) - if omitted, only plays beep; if provided, cannot be empty

<endcall> - End Call

End the call immediately.

{
  "action": "Thank you for your help <endcall />"
}

Attributes: None

Speech Control Tags

<silence> vs <hold> — both add pauses, but behave differently:

	`<silence>`	`<hold>`
Interruptible	✅ Yes — main agent can interrupt	❌ No — testing agent won’t be interrupted
After interruption	Condition matching restarts after main agent stops speaking	N/A
Background noise	Continues to play during the pause	Does not play during the pause
Idle prompt	Fires if the pause outlasts the personality’s idle timeout	Suppressed — the idle timer is paused for the hold

Use <silence> when you want the conversation to feel natural and allow the main agent to jump in. Use <hold> when you need a guaranteed delay before the next message.

<silence> - Add Pauses (interruptible)

Add a pause in speech. Interruptible — if the main agent speaks during the silence, the testing agent stops immediately and condition matching restarts once the main agent finishes. Background noise continues to play during the pause.

{
  "id": 3,
  "type": "standard",
  "condition": "The agent offers time slots",
  "action": "Let me think <silence time=\"2s\" /> I'll take the 3pm slot",
  "fixed_message": true
}

Attributes:

time: Duration of silence (required) - format: “Xs” where X is a number (e.g., “1s”, “2.5s”)

<hold> - Hold Between Messages (not interruptible)

Wait before sending the next message. Not interruptible — the testing agent will not be interrupted during a hold, regardless of what the main agent says. Background noise does not play during a hold, and the idle prompt (“Are you still there?”) is suppressed for its duration — so a hold can safely be longer than the personality’s idle timeout.

{
  "action": "Please wait <hold time=\"5s\" /> Are you still there?"
}

Can have multiple hold tags in one action:

{
  "action": "First part <hold time=\"2s\" /> Second part <hold time=\"3s\" /> Final part"
}

Attributes:

time: Duration to wait before next message (required) - format: “Xs” where X is a number (e.g., “5s”, ”10s”)

<spell> - Spell Out Text

Spell out text letter by letter (e.g., API → A—P—I).

{
  "action": "My reference code is <spell>ABC123</spell>"
}

Attributes: None - this is a wrapping tag that encloses the text to be spelled

<speed> - Control Speech Speed

Control the speed of speech (ratio: multiplier of base speed).

{
  "action": "<speed ratio=\"1.2\" /> I need to talk quickly about this"
}

Must be at the start of the message.Attributes:

ratio: Speed multiplier (required) - Valid range: 0.8 to 1.2
Examples: "1.2" for 20% faster, "0.8" for 20% slower

<volume> - Control Volume

Control the volume/loudness (ratio: multiplier of base volume).

{
  "action": "<volume ratio=\"1.5\" /> This is important!"
}

Must be at the start of the message.Attributes:

ratio: Volume multiplier (required) - Valid range: 0 to 2
Examples: "1.5" for 50% louder, "0.5" for 50% quieter

<voice> - Switch the Testing Agent's Voice

Switch the testing agent’s TTS voice mid-call. A self-closing tag without text changes every later utterance — including later conditions — until another persistent <voice> tag changes it. This is the persistent form.This is how you put more than one speaker in a single simulated call: a caller handing the phone to a family member, a supervisor taking over, or a different person picking up.

{
  "conditions": [
    {
      "id": 2,
      "condition": "agent asks to speak to the account holder",
      "action": "Hold on, let me get my husband. <voice provider=\"11labs\" id=\"21m00Tcm4TlvDq8ikWAM\" /> Hi, this is Mark speaking.",
      "type": "standard",
      "fixed_message": true
    }
  ]
}

The switch applies between utterances, so any text before the tag is spoken fully in the previous voice.Regional voice: Add text to speak only that text in the selected voice; the previous voice resumes immediately afterward.

{
  "action": "<voice provider=\"cartesia\" id=\"b7d50908-b17c-442d-ad8d-810c63997ed9\" text=\"Hi, this is the manager.\" /> How can I help?",
  "fixed_message": true
}

When regional speech contains inline tags such as <silence>, use an opening and closing block instead. Nested inline tags are allowed only inside this regional voice block.

{
  "action": "<voice provider=\"cartesia\" id=\"b7d50908-b17c-442d-ad8d-810c63997ed9\">Hi, this is the manager. <silence time=\"1s\" /> How can I help?</voice>",
  "fixed_message": true
}

Do not combine text with the opening/closing form. All <voice> tags require fixed_message: true.Attributes:

provider: Voice provider (required) - cartesia or 11labs. Must be the same provider the evaluator already runs on.
id: Voice ID (required) - must belong to that provider:
Provider Voice ID format Example
cartesia UUID b7d50908-b17c-442d-ad8d-810c63997ed9
11labs Alphanumeric, no dashes 21m00Tcm4TlvDq8ikWAM
model: TTS model (optional) - omit it to use the provider’s default: sonic-3.5 for Cartesia, eleven_turbo_v2_5 for ElevenLabs.

Provider	Voice ID format	Example
`cartesia`	UUID	`b7d50908-b17c-442d-ad8d-810c63997ed9`
`11labs`	Alphanumeric, no dashes	`21m00Tcm4TlvDq8ikWAM`

The provider cannot change mid-call. provider states which provider the voice ID belongs to so a mismatched pair is caught when you save the evaluator, rather than failing mid-call. To use a voice from a different provider, change the evaluator’s voice configuration instead.

Interaction Tags

<dtmf> - Send DTMF Tones

Send DTMF (touch-tone) digits.

{
  "id": 2,
  "type": "standard",
  "condition": "The agent asks for account number",
  "action": "<dtmf digits=\"123\" /> I've entered my account number",
  "fixed_message": true
}

Simulates pressing phone keypad buttons.Attributes:

digits: DTMF digits to send (required) - e.g., “123”, “456#”, “*9”

For caller data such as an account number or PIN, reference the test profile instead of hardcoding the digits — <dtmf digits="{{test_profile.customer_number}}#" />. The value is resolved per run and any formatting in it is stripped before the tones are sent. See DTMF testing.

<send_sms> - Send SMS

Trigger an SMS to be sent.

{
  "action": "<send_sms text=\"Confirmation code: 123456\" /> I've sent the code"
}

Attributes:

text: SMS message content to send (required)

<client_message> - Send an RTVI Client Message

Send an app-level RTVI client message to a Pipecat agent under test. Use this when the customer’s app would send an event such as opening a form, submitting data, or requesting session arguments during the conversation.

{
  "id": 2,
  "type": "standard",
  "condition": "The agent asks for the job number",
  "action": "<client_message t=\"get_session_args\" d='{\"job_number\":\"JB-42\"}' />",
  "fixed_message": true
}

The tag is silent: any text before or after it is spoken separately. Client messages are sent in the order they appear in the action.Attributes:

t: App-defined RTVI message type (required). Use the exact type your agent expects; Cekura does not invent or validate application-specific types.
d: Optional app-defined payload. JSON is decoded before it is sent. Use single quotes around the attribute when the JSON contains double quotes, as in the example above.

This tag is available for Pipecat agents that receive RTVI client messages. It requires fixed_message: true, like all tags in conditional actions.

<interruption> - Interrupt After Timeout

Interrupt the main agent after a specified time and deliver the message.

{
  "id": 2,
  "type": "action_followup",
  "condition": 1,
  "action": "<interruption time=\"3s\" /> Please listen carefully to these instructions",
  "fixed_message": true
}

The testing agent will interrupt the main agent after 3 seconds and speak this message.

<interruption> has two requirements:

The condition must have type: "action_followup"
The tag must appear at the very start of the action string

Attributes:

time: Duration to wait before interrupting (required) - format: “Xs” where X is a number (e.g., “3s”, “5.5s”)

Environmental Tags

<background_noise> - Background Sounds

Add continuous background sounds during a portion of speech.

{
  "action": "<background_noise sound=\"coffee-shop\" volume=\"0.3\">Hi, I'm calling about my order</background_noise>"
}

Attributes:

sound: Sound name (required) - see supported sounds below
volume: Volume level (optional) - default applies if not specified

Supported sounds:

Sound name	Description
`office-ambience`	Office environment
`coffee-shop`	Coffee shop chatter
`kitchen-noise`	Kitchen sounds
`home-chatter`	Home background voices
`vacuum-cleaner`	Vacuum cleaner
`dog-barking`	Dog barking
`baby-crying`	Baby crying
`keyboard-typing`	Keyboard typing
`coughing`	Coughing sounds
`background-printer`	Printer noise
`quiet-room`	Near-silent room
`air-conditioner`	AC hum
`construction-site`	Construction noise
`busy-street`	Street traffic
`airport-boarding`	Airport gate
`inside-car`	Moving car interior
`inside-train`	Moving train interior
`public-park`	Park ambience
`rain-thunder`	Rain and thunder
`windy-day`	Wind
`restaurant`	Restaurant ambience
`shopping-mall`	Shopping mall
`stadium-crowd`	Crowd noise
`standard-hiss`	White noise hiss
`static-radio`	Radio static
`fan-buzz`	Fan buzz
`ship-humming`	Low ship hum
`two-people-talking`	Background conversation
`train-station`	Train station hall
`holding-on-song`	Hold music

The sounds above are available via the <background_noise> tag in evaluator actions. To use a custom audio file URL as background noise (e.g., https://your-domain.com/noise.mp3), use the Custom URL option in the Background Noise dropdown when creating or editing a personality — no tag required.

<noise> - Play Sound Effects

Play a one-shot sound effect (does not loop).

{
  "action": "Hold on <noise sound=\"beep\" /> one moment"
}

Attributes:

sound: Sound name (required) - supported: office, beep, cough1, cough2
volume: Volume level (optional)
time: Duration in ms (optional)

<network_simulation> - Simulate Network Issues

Simulate packet loss on the network connection.

{
  "action": "<network_simulation packet_loss=\"5\" /> Testing under poor network"
}

Attributes:

packet_loss: Packet loss percentage (required) - e.g., "5" = 5% loss

Custom Functions

Functions let the testing agent fetch live data from your systems during the call and use it in what it says — look up an order, pull an account record, then respond with the real values instead of invented ones. Declare functions once in a top-level functions array (a sibling of role and conditions), then use them from any condition’s action.

{
  "role": "You are a customer checking on an order",
  "functions": [
    {
      "name": "lookup_order",
      "type": "rest_api",
      "auto_run": true,
      "config": {
        "method": "GET",
        "url": "https://api.example.com/orders/{{test_profile.order_id}}",
        "headers": { "Accept": "application/json" },
        "timeout_seconds": 5,
        "response_mapping": {
          "status": { "path": "$.data.status", "default": "unknown" },
          "eta_days": "$.data.eta"
        }
      }
    }
  ],
  "conditions": [ ... ]
}

Function fields

string

required

Unique name used in tags and placeholders. Letters, digits, _, -; up to 64 characters.

string

required

The function type. Currently rest_api — an HTTP call to your API.

boolean

default:"true"

When true (the default), the function runs once automatically at the start of the call, in the background — its values are ready before the conversation needs them. Set false to run the function only when a condition invokes it with a <function> tag.

string

default:"GET"

GET or POST.

string

required

The endpoint to call. Supports {{test_profile.*}} substitution. Must be an http(s) URL that is publicly reachable — internal and loopback addresses are refused at call time.

object

Request headers, e.g. { "Authorization": "Bearer ..." }. Values must be strings or numbers.

object

Query parameters appended to the URL. Values must be strings or numbers.

object | array | string

Request body (POST only). Objects and arrays are sent as JSON; strings are sent as-is.

number

default:"10"

Between 1 and 30 seconds. Keep it short — a slow endpoint delays the turn that’s waiting on it.

object

Names the values you want out of the JSON response — see Response mapping below.

When a function runs

Trigger	Runs	Choose it when
`auto_run: true` (default)	Once, at the start of the call, in the background	The data will be needed at some point in the call — it’s fetched before any turn waits on it
`<function name="lookup_order"/>` tag in an action	The moment that condition fires	The call must happen at a specific point in the flow (e.g. only after the agent asks), or you need to re-fetch — a tag always makes a fresh request

The tag is allowed on any condition except the First Message (id: 0), on both standard and action_followup conditions, fixed or non-fixed.

Using the response

Two ways to get fetched values into the testing agent’s replies:

Value placeholder — {{function.lookup_order.status}} (fixed messages only). Substitutes one mapped output into the exact spoken text:

{ "id": 2, "condition": 1, "action": "My order shows as {{function.lookup_order.status}}.", "type": "action_followup", "fixed_message": true }

Non-fixed actions — no placeholder needed. When fixed_message is false, the testing agent is automatically given the function’s results (your named outputs plus the response) and phrases its reply from them:
{ "id": 2, "condition": 1, "action": "Ask about the order and mention its real status.", "type": "action_followup", "fixed_message": false }

A single fixed action can both invoke and use a function — put the tag before the placeholder. The function completes before the text is spoken:"Let me check. <function name=\"lookup_order\"/> It shows as {{function.lookup_order.status}}."

Response mapping (JSONPath)

Each response_mapping entry gives a short output name to a value in the JSON response. An entry is either a path string, or an object with path and a default:

"response_mapping": {
  "status": "$.data.status",
  "eta_days": { "path": "$.data.eta", "default": "a few" }
}

Syntax	Meaning
`$.data.status`	Nested key
`$.items[0].sku`	Array index (negative indexes allowed)
`$.meta["order-id"]`	Bracket notation for keys with special characters

Wildcards, filters, and recursive descent are not supported — point each output at one exact value.

Chaining functions

A function’s config can reference an earlier function’s outputs, so one call can feed the next — for example, look up a customer, then verify with the fetched name:

"functions": [
  { "name": "lookup", "type": "rest_api",
    "config": { "method": "GET", "url": "https://api.example.com/orders/ORD-1",
      "response_mapping": { "customer": "$.customer_name" } } },
  { "name": "verify", "type": "rest_api", "auto_run": false,
    "config": { "method": "GET",
      "url": "https://api.example.com/verify?name={{function.lookup.customer}}",
      "response_mapping": { "verified": "$.ok" } } }
]

Make sure the referenced function has already run — via auto_run or a tag on an earlier condition.

If the call fails

A failed function never breaks the conversation. On a timeout, an error response, or an unreachable URL:

Mapped outputs fall back to their declared default.
A placeholder with no default is spoken as-is ("...status is {{function.lookup_order.status}}") — declare defaults for anything a fixed message references.
On non-fixed actions, the testing agent is told the lookup failed, so it doesn’t invent values.

Rules

Function names must be unique; <function> tags and {{function.*}} placeholders must reference a declared function.
Placeholders require fixed_message: true, and the key must be an output declared in that function’s response_mapping.
Neither tags nor placeholders may appear on the First Message (id: 0) — use auto_run and reference the values from a later condition.

For a full scenario showing both flows — a fixed-message placeholder and a non-fixed grounded reply from one auto_run function — see Example 5 below.

Attached Audio

Attach a pre-recorded audio clip to a condition and the testing agent plays the recording instead of speaking the text with TTS when that condition fires. Use it when the exact voice, accent, tone, or non-speech sound matters — reproducing a real caller’s recording, a specific IVR prompt, background audio a synthetic voice can’t recreate, or a regression case captured from a live call.

Audio can only be attached to a fixed-message condition (fixed_message: true). It is not available on instruction-based (fixed_message: false) conditions — those are meant to vary, and a recording is fixed by definition.

If you only need a second speaker rather than a specific recording, use the <voice> tag instead — it keeps the dialogue generated so the conversation can still adapt.

How it works

You upload an audio file to a specific condition (from the editor UI or the API).
Cekura transcribes it automatically in the background — this is why a clip is briefly processing before it’s ready to use.
Cekura inserts a managed <audio id="…"/> reference at the selected position in the action. You don’t write this tag by hand; attaching, replacing, and removing audio manages it for you.
When the condition fires, text, recordings, and tags run from left to right.

The transcript is derived from the recording and is read-only.

Attaching audio (UI)

In the Conditional Actions editor, each fixed-message condition has an Attach audio at cursor control:

Place the cursor where the recording should play, click Attach audio at cursor, and pick a file. You can attach more than one recording to the same action.
When processing finishes, each recording shows its read-only transcript and an audio player. Replace or Remove affects only that recording; surrounding text and tags stay in place.
While any attached clip is still processing (or has failed), runs are blocked for that scenario until it resolves — see Statuses & run gating.

Attaching audio (API)

Three endpoints on the scenario manage attached audio. The clip is uploaded as multipart/form-data.

# Attach a clip to condition 2 of scenario 39001
curl -X POST "https://api.cekura.ai/test_framework/v1/scenarios/39001/condition-audio/" \
  -H "Authorization: Bearer $CEKURA_API_KEY" \
  -F "condition_id=2" \
  -F 'action_template=Before the tone <audio /> <silence time="1s"/> continue' \
  -F "file=@greeting.mp3"
# 202 Accepted — transcription runs in the background

# List every attached clip on the scenario and its current status
curl "https://api.cekura.ai/test_framework/v1/scenarios/39001/condition-audio/" \
  -H "Authorization: Bearer $CEKURA_API_KEY"

# Detach and delete a clip by its audio_id
curl -X DELETE "https://api.cekura.ai/test_framework/v1/scenarios/39001/condition-audio/ab12cd34/" \
  -H "Authorization: Bearer $CEKURA_API_KEY"

The upload returns the clip’s entry immediately with status: "pending"; poll the list endpoint until it becomes ready. Each entry looks like:

{
  "audio_id": "ab12cd34",
  "condition_id": 2,
  "status": "ready",
  "transcript": "Hello, thanks for calling.",
  "duration": 1.78,
  "filename": "greeting.mp3",
  "playback_url": "https://…/ab12cd34.wav"
}

integer

required

The id of the condition to attach the clip to. The condition must already exist and have fixed_message: true. Save the scenario first — you attach to a saved condition.

file

required

The audio file. Max 25 MB; accepted formats: wav, mp3, m4a, ogg, webm, flac.

string

The complete action with one <audio /> marker where the new recording should be inserted. Existing managed audio tags must remain in the template.

string

To replace one recording, pass its audio_id and put <audio /> at that tag’s position in action_template.

Statuses & run gating

Each attached clip moves through these statuses:

Status	Meaning
`pending`	Uploaded; transcription queued.
`processing`	Transcribing.
`ready`	Transcript available; the clip will play on runs.
`failed`	Transcription failed. Retry (re-upload) or remove the clip.

A scenario cannot be run while any attached clip is not ready — the run is rejected with a clear message ("still transcribing — wait for it to finish" for pending/processing, "failed to transcribe — retry or remove it" for failed).

Complete Examples

Example 1: Appointment Cancellation

{
  "role": "You are a patient calling to cancel your appointment",
  "conditions": [
    {
      "id": 0,
      "condition": "FIRST_MESSAGE",
      "action": "Hello, I need to cancel my appointment",
      "type": "standard",
      "fixed_message": true
    },
    {
      "id": 1,
      "condition": "The agent asks for your name",
      "action": "Provide your name",
      "type": "standard",
      "fixed_message": false
    },
    {
      "id": 2,
      "condition": "The agent asks for your date of birth",
      "action": "Provide your date of birth",
      "type": "standard",
      "fixed_message": false
    },
    {
      "id": 3,
      "condition": "The agent confirms the cancellation",
      "action": "Thank them and end the call",
      "type": "standard",
      "fixed_message": false
    }
  ]
}

{
  "role": "You are a patient calling to cancel your appointment",
  "conditions": [
    {
      "id": 0,
      "condition": "FIRST_MESSAGE",
      "action": "Hi, I need to cancel my appointment on Tuesday",
      "type": "standard",
      "fixed_message": true
    },
    {
      "id": 1,
      "condition": "The agent offers to reschedule",
      "action": "Accept and say you prefer next week",
      "type": "standard",
      "fixed_message": false
    },
    {
      "id": 2,
      "condition": "The agent provides available time slots",
      "action": "Select the first available option",
      "type": "standard",
      "fixed_message": false
    },
    {
      "id": 3,
      "condition": "The agent asks if anything else is needed",
      "action": "No, that's all. Thank you for your help. <endcall />",
      "type": "standard",
      "fixed_message": true
    }
  ]
}

{
  "role": "You are a customer calling support",
  "conditions": [
    {
      "id": 0,
      "condition": "FIRST_MESSAGE",
      "action": "",
      "type": "standard",
      "fixed_message": true
    },
    {
      "id": 1,
      "condition": "The IVR plays the menu options",
      "action": "<dtmf digits=\"2\" />",
      "type": "standard",
      "fixed_message": true
    },
    {
      "id": 2,
      "condition": "The agent asks how they can help",
      "action": "Explain your billing issue",
      "type": "standard",
      "fixed_message": false
    },
    {
      "id": 3,
      "condition": "The agent asks for account number",
      "action": "<dtmf digits=\"123456#\" />",
      "type": "standard",
      "fixed_message": true
    }
  ]
}

{
  "role": "You are simulating a third-party IVR that the agent will call into",
  "conditions": [
    {
      "id": 0,
      "condition": "FIRST_MESSAGE",
      "action": "<ivr text=\"Thank you for calling Acme Corp. Press 1 for sales, press 2 for support.\" />",
      "type": "standard",
      "fixed_message": true
    },
    {
      "id": 1,
      "condition": "The main agent pressed 1",
      "action": "Connecting you to sales now. Please hold.",
      "type": "standard",
      "fixed_message": true
    },
    {
      "id": 2,
      "condition": "The main agent pressed 2",
      "action": "Connecting you to support now. Please hold.",
      "type": "standard",
      "fixed_message": true
    },
    {
      "id": 3,
      "condition": "The agent states their reason for calling",
      "action": "Thank you. Routing your call now. <endcall />",
      "type": "standard",
      "fixed_message": true
    }
  ]
}

Inbound vs outbound IVR patterns differ:

Inbound (main agent IS the IVR): Use <dtmf> to navigate the IVR menu played by the main agent. <dtmf> can appear in any condition.
Outbound (testing agent simulates an external IVR): Use <ivr text="..." /> to play the IVR menu. <ivr> can appear in any condition. When the scenario contains <ivr>, DTMF digits pressed by the main agent appear in the transcript — write your conditions using the specific digit (e.g., "The main agent pressed 1").

Example 3: Multi-Part Response with Followup

{
  "role": "You are a customer with a complex request",
  "conditions": [
    {
      "id": 0,
      "condition": "FIRST_MESSAGE",
      "action": "Hi, I need help with my order",
      "type": "standard",
      "fixed_message": true
    },
    {
      "id": 1,
      "type": "action_followup",
      "condition": 0,
      "action": "Also mention you haven't received a confirmation email",
      "fixed_message": false
    },
    {
      "id": 2,
      "type": "action_followup",
      "condition": 1,
      "action": "And ask if you can get expedited shipping",
      "fixed_message": false
    },
    {
      "id": 3,
      "condition": "The agent responds to your requests",
      "action": "Thank them and confirm the details",
      "type": "standard",
      "fixed_message": false
    }
  ]
}

Turn-by-turn flow for Example 3:

Testing agent sends condition 0: “Hi, I need help with my order”
Main agent replies
Action followup (condition: 0) fires — testing agent: “Also mention you haven’t received a confirmation email”
Main agent replies
Action followup (condition: 1) fires — testing agent: “And ask if you can get expedited shipping”
Main agent replies — condition 3 (standard) matches and fires

Example 4: Using Advanced Tags

{
  "role": "You are a customer in a noisy environment",
  "conditions": [
    {
      "id": 0,
      "condition": "FIRST_MESSAGE",
      "action": "<background_noise sound=\"coffee-shop\" volume=\"0.3\">Hi, I'm calling about my order</background_noise>",
      "type": "standard",
      "fixed_message": true
    },
    {
      "id": 1,
      "condition": "The agent asks you to repeat",
      "action": "<volume ratio=\"1.5\" />I said I'm calling about my order, <spell>ABC</spell> 123",
      "type": "standard",
      "fixed_message": true
    },
    {
      "id": 2,
      "condition": "The agent asks for more information",
      "action": "Let me check <silence time=\"2s\" /> here's the reference number",
      "type": "standard",
      "fixed_message": true
    }
  ]
}

Example 5: Live Data via a Custom Function

One auto_run function feeding both flows: condition 1 drops a fetched value into a fixed message with a {{function.*}} placeholder, and condition 2 lets the testing agent phrase a non-fixed reply from the same response — no placeholder needed.

{
  "role": "You are a customer checking on an order",
  "functions": [
    {
      "name": "lookup",
      "type": "rest_api",
      "auto_run": true,
      "config": {
        "method": "GET",
        "url": "https://api.example.com/orders/{{test_profile.order_id}}",
        "timeout_seconds": 5,
        "response_mapping": {
          "status": { "path": "$.status", "default": "unknown" },
          "customer": { "path": "$.customer_name", "default": "unknown" }
        }
      }
    }
  ],
  "conditions": [
    {
      "id": 0,
      "condition": "FIRST_MESSAGE",
      "action": "Hi, I'd like to check on my order.",
      "type": "standard",
      "fixed_message": true
    },
    {
      "id": 1,
      "condition": "The agent asks for the name on the order",
      "action": "It's under {{function.lookup.customer}}.",
      "type": "standard",
      "fixed_message": true
    },
    {
      "id": 2,
      "condition": 1,
      "action": "Ask whether the order status they see matches yours, mentioning the real status from the lookup.",
      "type": "action_followup",
      "fixed_message": false
    },
    {
      "id": 3,
      "condition": "The agent confirms the order status",
      "action": "Great, that's all I needed. Thanks! <endcall />",
      "type": "standard",
      "fixed_message": true
    }
  ]
}

Turn-by-turn flow for Example 5:

As the call connects, lookup runs once in the background (auto_run) — say it returns {"status": "shipped", "customer_name": "Jane Doe"}
Testing agent sends condition 0: “Hi, I’d like to check on my order.”
The agent asks for the name — condition 1 (fixed) fires with the placeholder substituted verbatim: “It’s under Jane Doe.”
The agent replies — condition 2 (non-fixed) fires; the testing agent is given the lookup results automatically and phrases them naturally: e.g. “On my side it shows as shipped — is that what you see?”
The agent confirms — condition 3 closes the call

If the API call fails, the declared defaults take over: condition 1 speaks “It’s under unknown.” and the testing agent in condition 2 is told the lookup failed rather than inventing a status. On-demand variant (auto_run: false): to run the lookup only at a specific point in the flow instead of at call start, turn off auto_run and invoke it with a tag — the function completes before the rest of the action is spoken:

"functions": [
  { "name": "lookup", "type": "rest_api", "auto_run": false,
    "config": { "method": "GET", "url": "https://api.example.com/orders/{{test_profile.order_id}}",
      "response_mapping": { "customer": { "path": "$.customer_name", "default": "unknown" } } } }
],
"conditions": [
  { "id": 1, "condition": "The agent asks for the name on the order",
    "action": "One moment. <function name=\"lookup\"/> It's under {{function.lookup.customer}}.",
    "type": "standard", "fixed_message": true }
]

Nothing runs at call start; the request fires the moment condition 1 executes, and the same fixed/non-fixed usage rules apply from there.

Best Practices

Clear Conditions

Write conditions that clearly describe when they should trigger.

✅ “The agent asks for your email”
❌ “Email”

Specific Actions

Make actions specific and actionable.

✅ “Provide your email address”
❌ “Answer the question”

Logical Flow

Order conditions in a logical conversation flow.Start with id: 0 (FIRST_MESSAGE), then follow natural conversation progression.

Common Patterns

Pattern 1: Conditional Branching

Handle different paths based on agent response:

{
  "conditions": [
    {
      "id": 0,
      "condition": "FIRST_MESSAGE",
      "action": "I would like to request a refund for my order",
      "type": "standard",
      "fixed_message": true
    },
    {
      "id": 1,
      "condition": "The agent approves the refund",
      "action": "Thank them and confirm the details",
      "type": "standard",
      "fixed_message": false
    },
    {
      "id": 2,
      "condition": "The agent denies the refund",
      "action": "Ask to speak with a manager",
      "type": "standard",
      "fixed_message": false
    }
  ]
}

Pattern 2: Information Gathering

Progressively provide information as requested:

{
  "conditions": [
    {
      "id": 0,
      "condition": "FIRST_MESSAGE",
      "action": "Hi, I'm calling about my account",
      "type": "standard",
      "fixed_message": true
    },
    {
      "id": 1,
      "condition": "The agent asks for your name",
      "action": "Provide your name",
      "type": "standard",
      "fixed_message": false
    },
    {
      "id": 2,
      "condition": "The agent asks for your account number",
      "action": "Provide your account number",
      "type": "standard",
      "fixed_message": false
    },
    {
      "id": 3,
      "condition": "The agent asks for verification",
      "action": "Provide your date of birth",
      "type": "standard",
      "fixed_message": false
    }
  ]
}

Pattern 3: Scripted Sequence with action_followup

Chain action_followup conditions to deliver an exact sequence of messages across turns, with no conditions to match — each followup fires automatically on the next turn after the main agent replies, regardless of what the main agent says:

{
  "conditions": [
    {
      "id": 0,
      "condition": "FIRST_MESSAGE",
      "action": "Hi, I need to update my address",
      "type": "standard",
      "fixed_message": true
    },
    {
      "id": 1,
      "type": "action_followup",
      "condition": 0,
      "action": "My new street is 123 Main Street",
      "fixed_message": true
    },
    {
      "id": 2,
      "type": "action_followup",
      "condition": 1,
      "action": "City is Springfield",
      "fixed_message": true
    },
    {
      "id": 3,
      "type": "action_followup",
      "condition": 2,
      "action": "Zip code is 62701",
      "fixed_message": true
    }
  ]
}

Tips

Start Simple: Begin with basic conditions and add complexity as needed. Test frequently.

Use Descriptive IDs: While IDs must be integers, use sequential numbering (0, 1, 2, …) for clarity. IDs must be unique across all conditions.

Test Edge Cases: Include conditions for unexpected agent responses or errors.

Troubleshooting

Validation error: first condition must have condition='FIRST_MESSAGE'

Issue: The API returns a validation error about the first condition.Solution: Ensure the first element in your conditions array has all three:

{ "id": 0, "condition": "FIRST_MESSAGE", "action": "...", "type": "standard", "fixed_message": true }

Previously condition was left empty (""); it must now be the exact string "FIRST_MESSAGE".

Validation error: type is required

Issue: The API returns a validation error about a missing type field.Solution: Add type explicitly to every condition. There is no default:

{ "id": 1, "condition": "Agent asks for name", "action": "Provide name", "type": "standard", "fixed_message": false }

Validation error: fixed_message is required

Issue: The API returns a validation error about a missing fixed_message field.Solution: Add fixed_message explicitly to every condition set to true or false:

{ "id": 1, "condition": "Agent asks for name", "action": "Provide name", "type": "standard", "fixed_message": false }

Validation error: duplicate condition ID

Issue: Two or more conditions share the same id.Solution: Ensure every condition has a unique integer id. Use sequential numbering (0, 1, 2, …) to avoid collisions.

Validation error: scenario_language is required

Issue: Creating a Conditional Actions evaluator fails with a language error.Solution: Assign a personality to the evaluator — the language is inferred automatically from the personality. Alternatively, set scenario_language explicitly in the request.

Condition Not Triggering

Issue: A condition isn’t triggering when expected.Solutions:

Make the condition more specific
Check if a previous condition is matching instead
Verify the condition describes what the agent says, not what the evaluator should do

action_followup firing unexpectedly or not at all

Issue: An action_followup fires at the wrong time or doesn’t fire.Explanation: action_followup fires on the next turn after the referenced condition — after the testing agent sends condition X and the main agent replies. It does not fire in the same turn as condition X.Solutions:

Ensure condition points to the correct ID of the preceding condition
If chaining multiple followups, each one references the ID of the previous followup (not the original condition)

Tags Not Working

Issue: Special tags aren’t producing the expected effect.Solutions:

Check tag syntax (spelling, attributes, closing format)
Check tag placement (some tags must be at the start or span the entire action)
Confirm fixed_message: true is set — tags don’t work with instruction-based actions

First Message Not Sending

Issue: The conversation doesn’t start.Solutions:

Ensure you have a condition with id: 0 and condition: "FIRST_MESSAGE"
Verify fixed_message: true is set on the first condition
Check that the role is defined
If the agent speaks first, set action to "" (empty string is allowed for FIRST_MESSAGE only)

Can't run — an attached audio clip is still transcribing / failed

Issue: Runs are blocked with a message about an attached audio clip.Solution: A scenario can’t run while any attached audio clip is not ready. Wait for transcription to finish (pending/processing), or if a clip failed, re-upload it or remove it. Check clip statuses via GET scenarios/{id}/condition-audio/.

Attach audio option is missing on a condition

Issue: A condition has no way to attach a recording.Solution: Audio attaches only to fixed-message conditions. Set fixed_message: true on the condition first.

Next Steps

Test Profiles

Add identity information to your conditional actions evaluator

Personalities

Configure language and behavioral characteristics

Metrics

Measure performance with custom metrics

Prompting Guide

Learn best practices for writing effective conditions

​What are Structured Tests?

​When to Use Structured Tests

Example Use Case

​Structure

​1. Role

​2. Conditions

​Condition Fields

​Validation Rules

​First Message (id: 0)

​Condition Types

​Standard Conditions

​Action Followup Conditions

​Action vs Fixed Message

​Using action with fixed_message: false

​Using action with fixed_message: true

​When to Use Each

Use fixed_message: false

Use fixed_message: true

​Supported Tags

​Communication Tags

​Speech Control Tags

​Interaction Tags

​Environmental Tags

​Custom Functions

​Function fields

​When a function runs

​Using the response

​Response mapping (JSONPath)

​Chaining functions

​If the call fails

​Rules

​Attached Audio

​How it works

​Attaching audio (UI)

​Attaching audio (API)

​Statuses & run gating

​Complete Examples

​Example 1: Appointment Cancellation

​Example 2: IVR Navigation

​Example 3: Multi-Part Response with Followup

​Example 4: Using Advanced Tags

​Example 5: Live Data via a Custom Function

​Best Practices

Clear Conditions

Specific Actions

Logical Flow

​Common Patterns

​Pattern 1: Conditional Branching

​Pattern 2: Information Gathering

​Pattern 3: Scripted Sequence with action_followup

​Tips

​Troubleshooting

​Next Steps

Test Profiles

Personalities

Metrics

Prompting Guide

What are Structured Tests?

When to Use Structured Tests

Structure

1. Role

2. Conditions

Condition Fields

Validation Rules

First Message (id: 0)

Condition Types

Standard Conditions

Action Followup Conditions

Action vs Fixed Message

Using `action` with `fixed_message: false`

Using `action` with `fixed_message: true`

When to Use Each

Supported Tags

Communication Tags

Speech Control Tags

Interaction Tags

Environmental Tags

Custom Functions

Function fields

When a function runs

Using the response

Response mapping (JSONPath)

Chaining functions

If the call fails

Rules

Attached Audio

How it works

Attaching audio (UI)

Attaching audio (API)

Statuses & run gating

Complete Examples

Example 1: Appointment Cancellation

Example 2: IVR Navigation

Example 3: Multi-Part Response with Followup

Example 4: Using Advanced Tags

Example 5: Live Data via a Custom Function

Best Practices

Common Patterns

Pattern 1: Conditional Branching

Pattern 2: Information Gathering

Pattern 3: Scripted Sequence with action_followup

Tips

Troubleshooting

Next Steps