> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cekura.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Structured Tests

> Create dynamic, rule-based evaluators that respond to specific conversation conditions

## What are Structured Tests?

Structured Tests is a specialized evaluator type that allows you to create dynamic, rule-based test scenarios. Unlike traditional evaluators that follow a linear script, Structured Tests evaluators can adapt their behavior based on what the agent says during the conversation.

This makes them ideal for testing complex workflows where the conversation can take multiple paths depending on the agent's responses.

Structured Tests are the default evaluator format. See [Evaluator Formats](https://docs.cekura.ai/documentation/key-concepts/evaluators/overview.md#evaluator-formats).

## When to Use Structured Tests

Use Structured Tests evaluators when you need to:

* **Test branching conversations**: Handle different conversation paths based on agent responses
* **Simulate decision trees**: Test scenarios where user responses depend on what the agent asks
* **Create adaptive evaluators**: Build evaluators that can handle unexpected agent responses
* **Test IVR (Interactive Voice Response) flows**: Simulate interactive voice response systems with menu options
* **Validate error handling**: Test how agents handle various user inputs and edge cases

> [card] Example Use Case
>
> Testing an appointment booking agent where the flow changes based on whether the agent offers available slots, asks for rescheduling, or cancels the appointment entirely.

## Structure

A Structured Tests evaluator has three main parts:

```json theme={null}
{
  "role": "You are a customer calling to cancel an appointment",
  "first_message": "Hi, I need to cancel my appointment",
  "conditions": [
    {
      "when": "The agent asks for your name",
      "say": "John Smith"
    },
    {
      "when": "The agent asks for a reason",
      "say": "<ai_generated>Say you have a scheduling conflict</ai_generated>"
    }
  ]
}
```

### 1. Role

The `role` defines who the testing agent is pretending to be.

> [!NOTE]
>
> Keep the role concise and clear. It sets the context for the entire conversation.

### 2. First message

`first_message` is what the testing agent says to open the call. It is always spoken word-for-word. Leave it empty (`""`) when the main agent speaks first.

### 3. Conditions

The `conditions` array defines the rules for the rest of the conversation. Each condition specifies:

* **When** a certain situation occurs (`when`)
* **What** the testing agent says (`say`)
* Optionally, what it says on the **following turns** (`then`)

## Changes from the earlier format

Structured tests used to be written as id-based rows. Responses now return the format on this page. The earlier format is deprecated: it is still accepted when you create or update an evaluator, so existing integrations keep working, but write new evaluators in the current format.

| Earlier format | Current format |
| - | - |
| Row with `"condition": "FIRST_MESSAGE"` (`id` 0) | `first_message` |
| `condition` | `when` |
| `action` | `say` |
| `action_followup` rows linked by `id` | `then` list on the condition |
| `fixed_message: true` | Plain step (spoken word-for-word) |
| `fixed_message: false` | Step wrapped in `<ai_generated>...</ai_generated>` |

```json Earlier theme={null}
{
  "role": "You are a patient calling to book an appointment",
  "conditions": [
    {"id": 0, "condition": "FIRST_MESSAGE", "action": "Hi, I need an appointment", "type": "standard", "fixed_message": true},
    {"id": 1, "condition": "The agent asks for your name", "action": "John Smith", "type": "standard", "fixed_message": true},
    {"id": 2, "condition": 1, "action": "<endcall />", "type": "action_followup", "fixed_message": true}
  ]
}
```

```json Current theme={null}
{
  "role": "You are a patient calling to book an appointment",
  "first_message": "Hi, I need an appointment",
  "conditions": [
    {"when": "The agent asks for your name", "say": "John Smith", "then": ["<endcall />"]}
  ]
}
```

## Fields

**role** `string` *required*

Who the testing agent is pretending to be.

**first\_message** `string | object`

What the testing agent says first, always word-for-word. Empty (or omitted) means the testing agent waits for the main agent to speak.

To add steps after the opening line, pass an object instead: `{"say": "...", "then": ["...", "..."]}`. `<ai_generated>` is not allowed in the first message.

**conditions** `array`

The ordered list of conditions. Each condition has:

**condition fields**

**when** `string` *required*

A third-person description of what the main agent does that triggers this condition — for example, `"The agent asks for your date of birth"`. Matched by meaning against the conversation.

**say** `string` *required*

The step the testing agent delivers when the condition matches. Spoken **word-for-word** by default. Wrap the whole step in `<ai_generated>...</ai_generated>` to have the testing agent improvise from it instead. Cannot be empty.

**then** `string[]`

Optional follow-up steps, in order. Each one fires on the testing agent's **next turn**, after one reply from the main agent — whatever that reply is. Each step follows the same rules as `say`.

**functions** `array`

Optional custom functions the testing agent can call during the conversation. See [Custom Functions](#custom-functions).

## Validation Rules

The backend enforces the following constraints on all Structured Tests evaluators. Violations return a validation error that names the exact place in your payload, such as `first_message`, `conditions[2].say` or `conditions[0].then[1]`.

> [accordion] 1. The first message is verbatim
>
> `first_message` (and `first_message.say`) is always spoken word-for-word, so it cannot contain `<ai_generated>`. Its `then` steps can.
>
> ```json theme={null}
> // ✅ Valid
> { "first_message": { "say": "Hello", "then": ["<ai_generated>Ask about your order</ai_generated>"] } }
>
> // ❌ Invalid — first_message: the first message is always said verbatim; <ai_generated> is not allowed.
> { "first_message": "<ai_generated>Greet the agent</ai_generated>" }
> ```

> [accordion] 2. Every condition needs `when` and a non-empty `say`
>
> `when` is required, and `say` cannot be empty or whitespace-only. Only `first_message` may be empty.
>
> ```json theme={null}
> // ✅ Valid
> { "when": "The agent greets you", "say": "Hi there" }
>
> // ❌ Invalid — conditions[0].when: is required.
> { "say": "Hi there" }
>
> // ❌ Invalid — conditions[0] has an empty step
> { "when": "The agent greets you", "say": "" }
> ```

> [accordion] 3. `\<ai_generated\>` wraps the whole step
>
> `<ai_generated>` must wrap the entire step. It cannot be partial, nested, or empty.
>
> ```json theme={null}
> // ✅ Valid
> { "when": "The agent asks about pricing", "say": "<ai_generated>Ask whether there is a student discount</ai_generated>" }
>
> // ❌ Invalid — conditions[0].say: <ai_generated> must wrap the whole step
> { "when": "The agent asks about pricing", "say": "Sure. <ai_generated>Ask about discounts</ai_generated>" }
> ```

> [accordion] 4. Only known fields
>
> A condition accepts only `when`, `say` and `then`; `first_message` as an object accepts only `say` and `then`. Unknown fields are rejected — for example, `conditions[0]: unknown field(s) fixed. Allowed: when, say, then.`

> [accordion] 5. Language required
>
> Structured Tests evaluators require a `scenario_language`. Set it by assigning a personality with a language configured. If you provide a personality, the language is inferred automatically; if no personality is set, `scenario_language` must be explicitly provided.
>
> This also applies when changing an existing evaluator's type to Structured Tests.

## First Message

`first_message` is the line the testing agent opens the call with:

```json theme={null}
{ "first_message": "Hi, I need to cancel my appointment" }
```

**When the main agent speaks first**, leave it empty:

```json theme={null}
{ "first_message": "" }
```

To keep talking after the opening line — whatever the main agent replies — add `then` steps:

```json theme={null}
{
  "first_message": {
    "say": "Hi, I need help with my order",
    "then": [
      "<ai_generated>Mention you haven't received a confirmation email</ai_generated>",
      "Can I also get expedited shipping?"
    ]
  }
}
```

> [!NOTE]
>
> The first message is sent **once** and is never retried. If the main agent's speech cuts it off, whatever had not played yet is dropped and never spoken later. A `<silence>` always gives way when the main agent starts talking, so a greeting that arrives during a leading `<silence>` drops the whole line; spoken text gives way according to the personality's interruption settings. The conversation then continues through the other conditions as usual. To let the main agent greet first and then say an exact line, leave `first_message` empty and put the line on a condition that matches the greeting:
>
> ```json theme={null}
> {
>   "first_message": "",
>   "conditions": [
>     { "when": "The agent greets the caller", "say": "Hi, I'm calling because my fare came out higher than the estimate." }
>   ]
> }
> ```

## Follow-up Steps (`then`)

A `then` step fires on the **next turn** after the step before it. It does not fire immediately. The sequence is:

1. The testing agent delivers the condition's `say`
2. The main agent receives it and replies
3. The first `then` step fires — the testing agent delivers it
4. The main agent replies, and the next `then` step fires, and so on

The main agent's reply is received but does not affect whether the follow-up fires. Because of this, `then` has two key uses:

* **Multi-part responses**: send several messages across consecutive turns after a single trigger
* **Scripted sequences**: deliver an exact sequence of messages from the testing agent, turn by turn, without needing any conditions to match — the sequence fires automatically regardless of what the main agent says

```json theme={null}
{
  "when": "The agent asks how they can help",
  "say": "I need to change my booking",
  "then": [
    "It's under the name Jordan Avery",
    "<ai_generated>Ask whether there is a change fee</ai_generated>"
  ]
}
```

> [!NOTE]
>
> Each condition has a single, linear list of `then` steps. A `then` step never fires in the same turn as the step before it — one reply from the main agent always comes in between.

## Verbatim vs AI-generated steps

Every step (`say` or a `then` entry) is one of two kinds:

### Verbatim (default)

The testing agent says the text exactly as written.

```json theme={null}
{ "when": "The agent asks for your name", "say": "My name is John Smith" }
```

The testing agent will say exactly: "My name is John Smith"

### AI-generated

Wrap the whole step in `<ai_generated>...</ai_generated>` and the text becomes an instruction the testing agent interprets to produce a natural reply.

```json theme={null}
{ "when": "The agent asks for your name", "say": "<ai_generated>Provide your name as John Smith</ai_generated>" }
```

The testing agent might say: "My name is John Smith" or "Sure, it's John Smith" or "John Smith"

### When to Use Each

> [card] Use \<ai\_generated>
>
> * Natural, varied responses
> * Testing agent adaptability
> * When exact wording doesn't matter

> [card] Use verbatim (default)
>
> * Exact phrases required
> * Testing specific keywords
> * Compliance testing
> * Reproducible test cases
> * The first message — always verbatim
> * Steps containing tags

## Supported Tags

Structured Tests supports a variety of special tags inside a step (`first_message`, `say`, or a `then` entry) to control conversation flow and behavior.

> [!WARNING]
>
> **Important**: Tags only work in **verbatim** steps. A step wrapped in `<ai_generated>` cannot contain tags — it is rejected at save time.

Tags can be combined with text and other sibling tags. They run from left to right; do not nest tags inside one another, except inside a regional `<voice ...>...</voice>` block. `<ivr>` and `<voicemail>` must be the whole step.

### Communication Tags

> [accordion] \<ivr> - Interactive Voice Response
>
> Mark a message as an IVR (Interactive Voice Response) message that cannot be interrupted.
>
> ```json theme={null}
> {
>   "when": "The call is answered",
>   "say": "<ivr text=\"Press 1 for sales, press 2 for support\" />"
> }
> ```
>
> The agent cannot interrupt while this message plays.
>
> > [!WARNING]
> >
> > `<ivr>` must be the **entire step** — it cannot be combined with other text or tags in the same step.
>
> > [!NOTE]
> >
> > **DTMF digits appear in the transcript when using `<ivr>`.** When a scenario contains the `<ivr>` tag, any DTMF digits pressed by the main agent (the agent being tested) are captured and appear in the transcript. This lets you write conditions that detect the exact digit pressed — for example, `"The main agent pressed 1"` — rather than relying on speech detection alone. Both `<ivr>` and `<dtmf>` can appear in any step, not just the first message.
>
> **Attributes:**
>
> * `text`: The IVR message to play (required)
>
> > [!WARNING]
> >
> > `<hold>` and `<audio>` tags are **not supported** inside `<ivr text="...">` and are rejected at save time. To play a sequence of audio clips and pauses without interruption, use `<ignore_interruptions>` instead.

> [accordion] \<ignore\_interruptions> - Protected Playback Span
>
> Wrap a span of the step so everything inside plays to completion — nothing the main agent says can interrupt it. Content is unrestricted: spoken text, `<audio>` clips, `<hold>`/`<silence>` pauses, or any mix. The typical use is a multi-part IVR menu of clips and pauses.
>
> ```json theme={null}
> {
>   "first_message": {
>     "say": "",
>     "then": [
>       "<ignore_interruptions> <audio id=\"2eeb6544\"/> <hold time=\"5s\" /> <audio id=\"f0c5cdfe\"/> <hold time=\"5s\" /> <audio id=\"1bfc2ad2\"/> </ignore_interruptions> <endcall />"
>     ]
>   }
> }
> ```
>
> Unlike `<ivr>`, this is a **block tag scoped to a span**: content goes between the opening and closing tags, it can appear more than once per step, and normal text or tags can come before and after it.
>
> **What the main agent's speech does during the span:**
>
> * It does **not** interrupt playback or abort the remaining clips.
>
> * It does **not** trigger a reply from the testing agent — during the span or after it ends.
>
> * It **is** still transcribed and included in evaluation, so metrics can judge what the agent said over the menu.
>
> > [!NOTE]
> >
> > Nested spans are not supported — use sequential spans instead. Attributes are not supported; content always goes between the tags.

> [accordion] \<voicemail> - Voicemail Greeting
>
> Play a voicemail greeting with beep.
>
> ```json theme={null}
> {
>   "first_message": "<voicemail text=\"Hi, you've reached John. Leave a message.\" />"
> }
> ```
>
> Or just a beep without greeting:
>
> ```json theme={null}
> {
>   "first_message": "<voicemail />"
> }
> ```
>
> > [!WARNING]
> >
> > `<voicemail>` must be the **entire step** — it cannot be combined with other text or tags in the same step.
>
> **Attributes:**
>
> * `text`: Voicemail greeting message (optional) - if omitted, only plays beep; if provided, cannot be empty

> [accordion] \<endcall> - End Call
>
> End the call immediately.
>
> ```json theme={null}
> {
>   "say": "Thank you for your help <endcall />"
> }
> ```
>
> **Attributes:** None

### Speech Control Tags

> [!NOTE]
>
> **`<silence>` vs `<hold>`** — both add pauses, but behave differently:
>
> | | `<silence>` | `<hold>` |
> | - | - | - |
> | Interruptible | ✅ Yes — main agent can interrupt | ❌ No — testing agent won’t be interrupted |
> | After interruption | Condition matching restarts after main agent stops speaking | N/A |
> | Background noise | Continues to play during the pause | Does **not** play during the pause |
> | [Idle prompt](https://docs.cekura.ai/documentation/key-concepts/evaluators/personality.md) | Fires if the pause outlasts the personality's idle timeout | Suppressed — the idle timer is paused for the hold |
>
> Use `<silence>` when you want the conversation to feel natural and allow the main agent to jump in. Use `<hold>` when you need a guaranteed delay before the next message.

> [accordion] \<silence> - Add Pauses (interruptible)
>
> Add a pause in speech. **Interruptible** — if the main agent speaks during the silence, the testing agent stops immediately and condition matching restarts once the main agent finishes. Background noise continues to play during the pause.
>
> ```json theme={null}
> {
>   "when": "The agent offers time slots",
>   "say": "Let me think <silence time=\"2s\" /> I'll take the 3pm slot"
> }
> ```
>
> **Attributes:**
>
> * `time`: Duration of silence (required) - format: "Xs" where X is a number (e.g., "1s", "2.5s")

> [accordion] \<hold> - Hold Between Messages (not interruptible)
>
> Wait before sending the next message. **Not interruptible** — the testing agent will not be interrupted during a hold, regardless of what the main agent says. Background noise does not play during a hold, and the idle prompt ("Are you still there?") is suppressed for its duration — so a hold can safely be longer than the personality's idle timeout.
>
> ```json theme={null}
> {
>   "say": "Please wait <hold time=\"5s\" /> Are you still there?"
> }
> ```
>
> Can have multiple hold tags in one action:
>
> ```json theme={null}
> {
>   "say": "First part <hold time=\"2s\" /> Second part <hold time=\"3s\" /> Final part"
> }
> ```
>
> **Attributes:**
>
> * `time`: Duration to wait before next message (required) - format: "Xs" where X is a number (e.g., "5s", "10s")

> [accordion] \<spell> - Spell Out Text
>
> Spell out text letter by letter (e.g., API → A—P—I).
>
> ```json theme={null}
> {
>   "say": "My reference code is <spell>ABC123</spell>"
> }
> ```
>
> **Attributes:** None - this is a wrapping tag that encloses the text to be spelled

> [accordion] \<speed> - Control Speech Speed
>
> Control the speed of speech (ratio: multiplier of base speed).
>
> ```json theme={null}
> {
>   "say": "<speed ratio=\"1.2\" /> I need to talk quickly about this"
> }
> ```
>
> The tag may appear anywhere in the step, and applies from that point until the
> next `<speed>` tag or the end of the step. Add `text` to scope it to just those
> words instead — the previous speed resumes afterwards:
>
> ```json theme={null}
> {
>   "say": "My date of birth is <speed ratio=\"1.8\" text=\"August fifth, nineteen seventy-nine\" />, got it?"
> }
> ```
>
> Speed is applied after synthesis and preserves pitch, so it works the same on
> every TTS provider. There is no `<speed>...</speed>` block form.
>
> **Attributes:**
>
> * `ratio`: Speed multiplier (required) - Valid range: **0.1 to 2.0**\\
>
>   0.8–1.2 keeps speech natural; beyond that it becomes a stress test.
> * `text`: Words to scope the ratio to (optional)\\
>
>   Cannot contain other tags — use the point form above if you need tags inside.

> [accordion] \<volume> - Control Volume
>
> Control the volume/loudness (ratio: multiplier of base volume).
>
> ```json theme={null}
> {
>   "say": "<volume ratio=\"1.5\" /> This is important!"
> }
> ```
>
> Like `<speed>`, it may appear anywhere and holds until the next `<volume>` tag or
> the end of the step, and `text` scopes it to just those words:
>
> ```json theme={null}
> {
>   "say": "My card number is <volume ratio=\"0.3\" text=\"four two one nine\" /> — did you catch that?"
> }
> ```
>
> Volume is applied as amplitude scaling after synthesis, so it works on every TTS
> provider. There is no `<volume>...</volume>` block form.
>
> **Attributes:**
>
> * `ratio`: Volume multiplier (required) - Valid range: **0 to 2**\\
>
>   Examples: `"1.5"` for 50% louder, `"0.5"` for 50% quieter. Above `1.0` the
>   output is clipped against the TTS signal, so prefer `<= 1.0` unless distortion
>   is what you are testing.
> * `text`: Words to scope the ratio to (optional)

> [accordion] \<voice> - Switch the Testing Agent's Voice
>
> Switch the testing agent's TTS voice mid-call. A self-closing tag without `text` changes
> every later utterance — including later steps and conditions — until another persistent `<voice>` tag
> changes it. This is the **persistent** form.
>
> This is how you put more than one speaker in a single simulated call: a caller handing the
> phone to a family member, a supervisor taking over, or a different person picking up.
>
> ```json theme={null}
> {
>   "conditions": [
>     {
>       "when": "The agent asks to speak to the account holder",
>       "say": "Hold on, let me get my husband. <voice provider=\"11labs\" id=\"21m00Tcm4TlvDq8ikWAM\" /> Hi, this is Mark speaking."
>     }
>   ]
> }
> ```
>
> The switch applies between utterances, so any text before the tag is spoken fully in the
> previous voice.
>
> **Regional voice:** Add `text` to speak only that text in the selected voice; the previous
> voice resumes immediately afterward.
>
> ```json theme={null}
> {
>   "say": "<voice provider=\"cartesia\" id=\"b7d50908-b17c-442d-ad8d-810c63997ed9\" text=\"Hi, this is the manager.\" /> How can I help?"
> }
> ```
>
> When regional speech contains inline tags such as `<silence>`, use an opening and closing
> block instead. Nested inline tags are allowed only inside this regional voice block.
>
> ```json theme={null}
> {
>   "say": "<voice provider=\"cartesia\" id=\"b7d50908-b17c-442d-ad8d-810c63997ed9\">Hi, this is the manager. <silence time=\"1s\" /> How can I help?</voice>"
> }
> ```
>
> Do not combine `text` with the opening/closing form. All `<voice>` tags require
> a verbatim step.
>
> **Attributes:**
>
> * `provider`: Voice provider (required) - `cartesia` or `11labs`. Must be the **same provider
>   the evaluator already runs on**.
>
> * `id`: Voice ID (required) - must belong to that provider:
>
>   | Provider | Voice ID format | Example |
>   | - | - | - |
>   | `cartesia` | UUID | `b7d50908-b17c-442d-ad8d-810c63997ed9` |
>   | `11labs` | Alphanumeric, no dashes | `21m00Tcm4TlvDq8ikWAM` |
>
> * `model`: TTS model (optional) - omit it to use the provider's default:
>   `sonic-3.5` for Cartesia, `eleven_turbo_v2_5` for ElevenLabs.
>
> > [!WARNING]
> >
> > **The provider cannot change mid-call.** `provider` states which provider the voice ID
> > belongs to so a mismatched pair is caught when you save the evaluator, rather than failing
> > mid-call. To use a voice from a different provider, change the evaluator's voice
> > configuration instead.

### Interaction Tags

> [accordion] \<dtmf> - Send DTMF Tones
>
> Send DTMF (touch-tone) digits.
>
> ```json theme={null}
> {
>   "when": "The agent asks for your account number",
>   "say": "<dtmf digits=\"123\" /> I've entered my account number"
> }
> ```
>
> Simulates pressing phone keypad buttons.
>
> **Attributes:**
>
> * `digits`: DTMF digits to send (required) - e.g., "123", "456#", "\*9"
>
> For caller data such as an account number or PIN, reference the test profile
> instead of hardcoding the digits — `<dtmf digits="{{test_profile.customer_number}}#" />`.
> The value is resolved per run and any formatting in it is stripped before the
> tones are sent. See [DTMF testing](https://docs.cekura.ai/documentation/guides/testing-agents/dtmf-testing.md).

> [accordion] \<send\_sms> - Send SMS
>
> Trigger an SMS to be sent.
>
> ```json theme={null}
> {
>   "say": "<send_sms text=\"Confirmation code: 123456\" /> I've sent the code"
> }
> ```
>
> **Attributes:**
>
> * `text`: SMS message content to send (required)

> [accordion] \<client\_message> - Send an RTVI Client Message
>
> Send an app-level [RTVI](https://github.com/rtvi-ai) client message to a
> Pipecat agent under test. Use this when the customer's app would send an
> event such as opening a form, submitting data, or requesting session
> arguments during the conversation.
>
> ```json theme={null}
> {
>   "when": "The agent asks for the job number",
>   "say": "<client_message t=\"get_session_args\" d='{\"job_number\":\"JB-42\"}' />"
> }
> ```
>
> The tag is silent: any text before or after it is spoken separately. Client
> messages are sent in the order they appear in the step.
>
> **Attributes:**
>
> * `t`: App-defined RTVI message type (required). Use the exact type your
>   agent expects; Cekura does not invent or validate application-specific types.
>
> * `d`: Optional app-defined payload. JSON is decoded before it is sent. Use
>   single quotes around the attribute when the JSON contains double quotes,
>   as in the example above.
>
> > [!WARNING]
> >
> > This tag is available for Pipecat agents that receive RTVI client messages.
> > It requires a verbatim step, like all tags in Structured Tests.

> [accordion] \<interruption> - Interrupt After Timeout
>
> Interrupt the main agent after a specified time and deliver the message.
>
> ```json theme={null}
> {
>   "when": "The agent starts explaining the cancellation policy",
>   "say": "Okay.",
>   "then": ["<interruption time=\"3s\" /> Please listen carefully to these instructions"]
> }
> ```
>
> The testing agent will interrupt the main agent after 3 seconds and speak this message.
>
> > [!WARNING]
> >
> > `<interruption>` has three requirements:
> >
> > 1. It can only be used in a `then` step (including `first_message.then`) — not in `first_message` or a condition's `say`
> > 2. The tag must appear at the **very start** of the step
> > 3. Something audible must follow it: spoken text, or a sound clip (`<noise>` or `<audio>`). A bare tag, or one followed only by `<silence>` or `<hold>`, is rejected
>
> To cut in with a sound and no words, for example a cough over the agent:
>
> ```json theme={null}
> {
>   "when": "The agent starts reading the terms",
>   "say": "Sure, go ahead.",
>   "then": ["<interruption time=\"1.2s\" /> <noise sound=\"cough1\" />"]
> }
> ```
>
> **Attributes:**
>
> * `time`: Duration to wait before interrupting (required) - format: "Xs" where X is a number (e.g., "3s", "5.5s")

### Environmental Tags

> [accordion] \<background\_noise> - Background Sounds
>
> Add continuous background sounds during a portion of speech.
>
> ```json theme={null}
> {
>   "say": "<background_noise sound=\"coffee-shop\" volume=\"0.3\">Hi, I'm calling about my order</background_noise>"
> }
> ```
>
> **Attributes:**
>
> * `sound`: Sound name (required) - see supported sounds below
> * `volume`: Volume level (optional) - between **0 and 1.0**; default applies if not specified
>
> **Supported sounds:**
>
> | Sound name | Description |
> | - | - |
> | `office-ambience` | Office environment |
> | `coffee-shop` | Coffee shop chatter |
> | `kitchen-noise` | Kitchen sounds |
> | `home-chatter` | Home background voices |
> | `vacuum-cleaner` | Vacuum cleaner |
> | `dog-barking` | Dog barking |
> | `baby-crying` | Baby crying |
> | `keyboard-typing` | Keyboard typing |
> | `coughing` | Coughing sounds |
> | `background-printer` | Printer noise |
> | `quiet-room` | Near-silent room |
> | `air-conditioner` | AC hum |
> | `construction-site` | Construction noise |
> | `busy-street` | Street traffic |
> | `airport-boarding` | Airport gate |
> | `inside-car` | Moving car interior |
> | `inside-train` | Moving train interior |
> | `public-park` | Park ambience |
> | `rain-thunder` | Rain and thunder |
> | `windy-day` | Wind |
> | `restaurant` | Restaurant ambience |
> | `shopping-mall` | Shopping mall |
> | `stadium-crowd` | Crowd noise |
> | `standard-hiss` | White noise hiss |
> | `static-radio` | Radio static |
> | `fan-buzz` | Fan buzz |
> | `ship-humming` | Low ship hum |
> | `two-people-talking` | Background conversation |
> | `train-station` | Train station hall |
> | `holding-on-song` | Hold music |
>
> > [!NOTE]
> >
> > The sounds above are available via the `<background_noise>` tag in evaluator steps. To use a **custom audio file URL** as background noise (e.g., `https://your-domain.com/noise.mp3`), use the **Custom URL** option in the Background Noise dropdown when [creating or editing a personality](https://docs.cekura.ai/documentation/advanced/creating-custom-personalities.md) — no tag required.

> [accordion] \<noise> - Play Sound Effects
>
> Play a one-shot sound effect (does not loop).
>
> ```json theme={null}
> {
>   "say": "Hold on <noise sound=\"beep\" /> one moment"
> }
> ```
>
> **Attributes:**
>
> * `sound`: Sound name (required) - a name from the sound library below, or a direct `https://` URL to an audio file
> * `volume`: Volume level (optional) - between **0 and 1.0**
> * `time`: Duration in ms (optional)
>
> **Sound library**: `office`, `beep`, `cough1`, `cough2`, `female-crying`, `male-crying`, `ringback`. The library grows over time - newly added sounds work with the same syntax as soon as they are available.
>
> `ringback` is \~6 seconds of a phone ringback tone (one ring, the gap, and the start of the next). The crying sounds are \~10 seconds of a person sobbing. Use them to check whether your agent notices a caller in distress and asks if everything is okay; shorten with `time` if needed.
>
> ```json theme={null}
> {
>   "say": "<noise sound=\"female-crying\" time=\"6000\" /> Sorry... I just got some bad news."
> }
> ```

> [accordion] \<network\_simulation> - Simulate Network Issues
>
> Simulate packet loss on the network connection.
>
> ```json theme={null}
> {
>   "say": "<network_simulation packet_loss=\"5\" /> Testing under poor network"
> }
> ```
>
> **Attributes:**
>
> * `packet_loss`: Packet loss percentage (required) - e.g., `"5"` = 5% loss

## Custom Functions

Functions let the testing agent fetch **live data from your systems during the call** and use it in what it says — look up an order, pull an account record, then respond with the real values instead of invented ones. Declare functions once in a top-level `functions` array (a sibling of `role` and `conditions`), then use them from any step after the first message.

```json theme={null}
{
  "role": "You are a customer checking on an order",
  "functions": [
    {
      "name": "lookup_order",
      "type": "rest_api",
      "auto_run": true,
      "config": {
        "method": "GET",
        "url": "https://api.example.com/orders/{{test_profile.order_id}}",
        "headers": { "Accept": "application/json" },
        "timeout_seconds": 5,
        "response_mapping": {
          "status": { "path": "$.data.status", "default": "unknown" },
          "eta_days": "$.data.eta"
        }
      }
    }
  ],
  "conditions": [ ... ]
}
```

### Function fields

**name** `string` *required*

Unique name used in tags and placeholders. Letters, digits, `_`, `-`; up to 64 characters.

**type** `string` *required*

The function type. Currently `rest_api` — an HTTP call to your API.

**auto\_run** `boolean` (default: true)

When `true` (the default), the function runs once automatically at the start of the call, in the background — its values are ready before the conversation needs them. Set `false` to run the function only when a condition invokes it with a `<function>` tag.

**config.method** `string` (default: GET)

`GET` or `POST`.

**config.url** `string` *required*

The endpoint to call. Supports `{{test_profile.*}}` substitution. Must be an `http(s)` URL that is **publicly reachable** — internal and loopback addresses are refused at call time.

**config.headers** `object`

Request headers, e.g. `{ "Authorization": "Bearer ..." }`. Values must be strings or numbers.

**config.query\_params** `object`

Query parameters appended to the URL. Values must be strings or numbers.

**config.body** `object | array | string`

Request body (`POST` only). Objects and arrays are sent as JSON; strings are sent as-is.

**config.timeout\_seconds** `number` (default: 10)

Between 1 and 30 seconds. Keep it short — a slow endpoint delays the turn that's waiting on it.

**config.response\_mapping** `object`

Names the values you want out of the JSON response — see [Response mapping](#response-mapping-jsonpath) below.

### When a function runs

| Trigger | Runs | Choose it when |
| - | - | - |
| `auto_run: true` (default) | Once, at the start of the call, in the background | The data will be needed at some point in the call — it's fetched before any turn waits on it |
| `<function name="lookup_order"/>` tag in a step | The moment that condition fires | The call must happen at a specific point in the flow (e.g. only after the agent asks), or you need to **re-fetch** — a tag always makes a fresh request |

The tag is allowed in any step except the first message itself — a condition's `say` or any `then` step (including `first_message.then`), verbatim or `<ai_generated>`.

### Using the response

Two ways to get fetched values into the testing agent's replies:

* **Value placeholder — `{{function.lookup_order.status}}`** (verbatim steps only). Substitutes one mapped output into the exact spoken text:

  ```json theme={null}
  { "when": "The agent asks about your order", "say": "My order shows as {{function.lookup_order.status}}." }
  ```

* **AI-generated steps — no placeholder needed.** In an `<ai_generated>` step, the testing agent is automatically given the function's results (your named outputs plus the response) and phrases its reply from them:

  ```json theme={null}
  { "when": "The agent asks about your order", "say": "<ai_generated>Ask about the order and mention its real status.</ai_generated>" }
  ```

> [!NOTE]
>
> A single verbatim step can both invoke and use a function — put the tag **before** the placeholder. The function completes before the text is spoken:
>
> `"Let me check. <function name=\"lookup_order\"/> It shows as {{function.lookup_order.status}}."`

### Response mapping (JSONPath)

Each `response_mapping` entry gives a short output name to a value in the JSON response. An entry is either a path string, or an object with `path` and a `default`:

```json theme={null}
"response_mapping": {
  "status": "$.data.status",
  "eta_days": { "path": "$.data.eta", "default": "a few" }
}
```

| Syntax | Meaning |
| - | - |
| `$.data.status` | Nested key |
| `$.items[0].sku` | Array index (negative indexes allowed) |
| `$.meta["order-id"]` | Bracket notation for keys with special characters |

Wildcards, filters, and recursive descent are **not** supported — point each output at one exact value.

### Chaining functions

A function's `config` can reference an earlier function's outputs, so one call can feed the next — for example, look up a customer, then verify with the fetched name:

```json theme={null}
"functions": [
  { "name": "lookup", "type": "rest_api",
    "config": { "method": "GET", "url": "https://api.example.com/orders/ORD-1",
      "response_mapping": { "customer": "$.customer_name" } } },
  { "name": "verify", "type": "rest_api", "auto_run": false,
    "config": { "method": "GET",
      "url": "https://api.example.com/verify?name={{function.lookup.customer}}",
      "response_mapping": { "verified": "$.ok" } } }
]
```

Make sure the referenced function has already run — via `auto_run` or a tag on an earlier condition.

### If the call fails

A failed function never breaks the conversation. On a timeout, an error response, or an unreachable URL:

* Mapped outputs fall back to their declared `default`.
* A placeholder with **no** default is spoken as-is (`"...status is {{function.lookup_order.status}}"`) — declare defaults for anything a verbatim step references.
* In `<ai_generated>` steps, the testing agent is told the lookup failed, so it doesn't invent values.

### Rules

* Function `name`s must be unique; `<function>` tags and `{{function.*}}` placeholders must reference a declared function.
* Placeholders require a verbatim step, and the key must be an output declared in that function's `response_mapping`.
* Neither tags nor placeholders may appear in the first message itself — use `auto_run` and reference the values from a later step.

For a full scenario showing both flows — a verbatim-step placeholder and an AI-generated grounded reply from one `auto_run` function — see [Example 5](#example-5-live-data-via-a-custom-function) below.

## Attached Audio

Attach a pre-recorded audio clip to a scenario and the testing agent **plays the recording instead of speaking the text with TTS** when a referencing condition fires. A single clip can be referenced from any number of conditions and even multiple times within one step. Use it when the exact voice, accent, tone, or non-speech sound matters — reproducing a real caller's recording, a specific IVR prompt, background audio a synthetic voice can't recreate, or a regression case captured from a live call.

> [!NOTE]
>
> Audio can only be used in a **verbatim** step. It is not available in `<ai_generated>` steps — those are meant to vary, and a recording is fixed by definition.

> [!TIP]
>
> If you only need a **second speaker** rather than a specific recording, use the [`<voice>` tag](#supported-tags) instead — it keeps the dialogue generated so the conversation can still adapt.

### How it works

1. You upload an audio file to the scenario, optionally attaching it to a condition (from the editor UI or the API). Each clip gets a human-readable **name** that becomes its id — for example, `greeting` or `hold-music`.
2. Cekura **transcribes it automatically** in the background — this is why a clip is briefly *processing* before it's ready to use.
3. Reference the clip from any number of verbatim steps by pasting its tag: `<audio id="greeting"/>`. You can write this tag by hand, or let the upload endpoint insert it for you when you pass `condition_id`.
4. When the condition fires, text, recordings, and tags run from left to right. The same clip plays wherever it is referenced.

> [!WARNING]
>
> The transcript is **derived from the recording** automatically. In the editor UI it is read-only; to correct a misheard word through the API, see [Editing a clip's transcript](#editing-a-clips-transcript).

### Attaching audio (UI)

In the Structured Tests editor, each verbatim step has an **Attach audio at cursor** control:

1. Place the cursor where the recording should play, click **Attach audio at cursor**, and pick a file. You can attach more than one recording to the same step.
2. When processing finishes, each recording shows its **read-only transcript** and an audio player. **Replace** or **Remove** affects only that recording; surrounding text and tags stay in place.
3. While any attached clip is still processing (or has failed), **runs are blocked** for that scenario until it resolves — see [Statuses & run gating](#statuses-run-gating).

### Attaching audio (API)

Five endpoints on the scenario manage attached audio. Clips are uploaded as `multipart/form-data`, or as a JSON body with the file base64-encoded (see `file_name` and `content_base64` below).

```bash Upload (with condition) theme={null}
# Attach a clip named "greeting" to step 2 of scenario 39001
curl -X POST "https://api.cekura.ai/test_framework/v1/scenarios/39001/condition-audio/" \
  -H "X-CEKURA-API-KEY: $CEKURA_API_KEY" \
  -F "name=greeting" \
  -F "condition_id=2" \
  -F 'action_template=Before the tone <audio /> <silence time="1s"/> continue' \
  -F "file=@greeting.mp3"
# 202 Accepted — transcription runs in the background
```

```bash Upload (without condition) theme={null}
# Add a clip to the scenario without touching any step
curl -X POST "https://api.cekura.ai/test_framework/v1/scenarios/39001/condition-audio/" \
  -H "X-CEKURA-API-KEY: $CEKURA_API_KEY" \
  -F "name=hold-music" \
  -F "file=@hold-music.wav"
# Then reference it from any verbatim step: <audio id="hold-music"/>
```

```bash List theme={null}
# List every clip on the scenario with its current status and references
curl "https://api.cekura.ai/test_framework/v1/scenarios/39001/condition-audio/" \
  -H "X-CEKURA-API-KEY: $CEKURA_API_KEY"
```

```bash Replace theme={null}
# Replace a clip's audio (every reference plays the new recording)
curl -X PUT "https://api.cekura.ai/test_framework/v1/scenarios/39001/condition-audio/greeting/" \
  -H "X-CEKURA-API-KEY: $CEKURA_API_KEY" \
  -F "file=@greeting-v2.mp3"
# Pass name=new-name to rename the clip at the same time
```

```bash Edit transcript theme={null}
# Correct a ready clip's transcript; the audio is untouched
curl -X PATCH "https://api.cekura.ai/test_framework/v1/scenarios/39001/condition-audio/greeting/" \
  -H "X-CEKURA-API-KEY: $CEKURA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"transcript": "Hello, John Smith."}'
```

```bash Delete theme={null}
# Delete a clip and remove every <audio> reference to it
curl -X DELETE "https://api.cekura.ai/test_framework/v1/scenarios/39001/condition-audio/greeting/" \
  -H "X-CEKURA-API-KEY: $CEKURA_API_KEY"
# Returns 409 if deleting would leave a step other than the first message empty
```

The upload returns the clip's entry immediately with `status: "pending"`; poll the list endpoint until it becomes `ready`. Each entry — from the list, upload, replace and edit responses — looks like:

```json theme={null}
{
  "audio_id": "greeting",
  "condition_ids": [2, 5],
  "steps": ["conditions[0].then[0]", "conditions[2]"],
  "tag": "<audio id=\"greeting\"/>",
  "status": "ready",
  "transcript": "Hello, thanks for calling.",
  "duration": 1.78,
  "words": [
    {"word": "hello", "start": 0.12, "end": 0.48},
    {"word": "thanks", "start": 0.56, "end": 0.9},
    {"word": "for", "start": 0.9, "end": 1.04},
    {"word": "calling", "start": 1.04, "end": 1.6}
  ],
  "timing_source": "deepgram",
  "filename": "greeting.mp3",
  "playback_url": "https://…/greeting.wav"
}
```

`steps` lists every place the clip is used, as paths into the evaluator (`first_message`, `conditions[1]`, `conditions[0].then[0]`), alongside the numeric `condition_ids`.

The scenario's GET response also includes a read-only `condition_audio` summary with every clip's tag and where it is used, so an author editing `instructions` directly can discover which clip ids exist:

```json theme={null}
"condition_audio": {
  "greeting": {
    "tag": "<audio id=\"greeting\"/>",
    "status": "ready",
    "condition_ids": [0, 3],
    "steps": ["first_message", "conditions[1].then[0]"]
  }
}
```

`condition_ids` (and the upload's `condition_id`) number the steps from 0 in reading order: `first_message` is 0, followed by its `then` steps, then each condition's `say` followed by its `then` steps.

**name** `string`

The clip's id, used in its `<audio>` tag and as its stored object name. Letters, digits, underscores and hyphens, max 64 characters; must be unique within the scenario (case-insensitive). Omit to derive a readable name from the filename (lowercased, hyphenated, trimmed to 32 characters).

**condition\_id** `integer`

The number of the step to attach the clip to, counted from 0 in reading order (see above). The step must already exist and be verbatim. Omit to add the clip to the scenario without touching any step — then reference it yourself with `<audio id="..."/>` from as many steps as needed.

**file** `file` *required*

The audio file. Max **25 MB**; accepted formats: `wav`, `mp3`, `m4a`, `ogg`, `webm`, `flac`.

**file\_name** `string`

JSON bodies only, in place of `file`. The clip's filename including extension, for example `greeting.mp3`.

**content\_base64** `string`

JSON bodies only, in place of `file`. The audio file as base64, either raw or as a `data:audio/mpeg;base64,...` data URL. The same size and format limits apply.

**action\_template** `string`

Requires `condition_id`. The complete step text with one `<audio />` marker where the new recording should be inserted. Existing managed audio tags must remain in the template.

**replace\_audio\_id** `string`

Requires `condition_id`. Points that step's existing tag at the new clip. To swap a clip's audio while keeping every reference to it, PUT to `condition-audio/{audio_id}/` instead.

### Editing a clip's transcript

`PATCH` on a clip corrects its `transcript` and, optionally, its word timings (`words`). The testing agent records the corrected text as what it said when the clip plays.

* **`words` sent**: stored as given. They must spell out the transcript (case, spacing and punctuation are ignored), be in playback order, have each `end` at or after its `start`, and end within the clip's duration. At most 5,000 words of up to 100 characters each; otherwise the request returns 400.
* **`words` omitted, same word count**: every word keeps its current timing.
* **`words` omitted, different word count**: the new words are spread across the span the old ones covered.
* Only a `ready` clip can be edited. A pending, processing or failed clip returns 409.
* Replacing the clip's audio transcribes it again and discards the edit.
* Runs already in progress are unaffected; the edit applies from the next run.

The response is the clip's entry, with `timing_source` showing where its timings came from: `deepgram` (measured), `interpolated` (spread evenly) or `manual` (sent with an edit).

```bash theme={null}
# Correct the transcript and send the word timings with it
curl -X PATCH "https://api.cekura.ai/test_framework/v1/scenarios/39001/condition-audio/greeting/" \
  -H "X-CEKURA-API-KEY: $CEKURA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"transcript": "Hello, John Smith.",
       "words": [{"word": "Hello,", "start": 0.12, "end": 0.48},
                 {"word": "John", "start": 0.6, "end": 0.9},
                 {"word": "Smith.", "start": 0.95, "end": 1.5}]}'
```

### Statuses & run gating

Each attached clip moves through these statuses:

| Status | Meaning |
| - | - |
| `pending` | Uploaded; transcription queued. |
| `processing` | Transcribing. |
| `ready` | Transcript available; the clip will play on runs. |
| `failed` | Transcription failed. Retry (re-upload) or remove the clip. |

A scenario **cannot be run while any attached clip is not `ready`** — the run is rejected with a clear message (`"still transcribing — wait for it to finish"` for pending/processing, `"failed to transcribe — retry or remove it"` for failed).

## Complete Examples

### Example 1: Appointment Cancellation

```json Basic Cancellation theme={null}
{
  "role": "You are a patient calling to cancel your appointment",
  "first_message": "Hello, I need to cancel my appointment",
  "conditions": [
    {
      "when": "The agent asks for your name",
      "say": "<ai_generated>Provide your name</ai_generated>"
    },
    {
      "when": "The agent asks for your date of birth",
      "say": "<ai_generated>Provide your date of birth</ai_generated>"
    },
    {
      "when": "The agent confirms the cancellation",
      "say": "<ai_generated>Thank them and say goodbye</ai_generated>"
    }
  ]
}
```

```json With Reschedule Option theme={null}
{
  "role": "You are a patient calling to cancel your appointment",
  "first_message": "Hi, I need to cancel my appointment on Tuesday",
  "conditions": [
    {
      "when": "The agent offers to reschedule",
      "say": "<ai_generated>Accept and say you prefer next week</ai_generated>"
    },
    {
      "when": "The agent provides available time slots",
      "say": "<ai_generated>Select the first available option</ai_generated>"
    },
    {
      "when": "The agent asks if anything else is needed",
      "say": "No, that's all. Thank you for your help. <endcall />"
    }
  ]
}
```

### Example 2: IVR Navigation

```json Inbound IVR (main agent is the IVR) theme={null}
{
  "role": "You are a customer calling support",
  "first_message": "",
  "conditions": [
    {
      "when": "The IVR plays the menu options",
      "say": "<dtmf digits=\"2\" />"
    },
    {
      "when": "The agent asks how they can help",
      "say": "<ai_generated>Explain your billing issue</ai_generated>"
    },
    {
      "when": "The agent asks for your account number",
      "say": "<dtmf digits=\"123456#\" />"
    }
  ]
}
```

```json Outbound IVR (testing agent simulates an external IVR) theme={null}
{
  "role": "You are simulating a third-party IVR that the agent will call into",
  "first_message": "<ivr text=\"Thank you for calling Acme Corp. Press 1 for sales, press 2 for support.\" />",
  "conditions": [
    {
      "when": "The main agent pressed 1",
      "say": "Connecting you to sales now. Please hold."
    },
    {
      "when": "The main agent pressed 2",
      "say": "Connecting you to support now. Please hold."
    },
    {
      "when": "The agent states their reason for calling",
      "say": "Thank you. Routing your call now. <endcall />"
    }
  ]
}
```

> [!NOTE]
>
> **Inbound vs outbound IVR patterns differ:**
>
> * **Inbound** (main agent IS the IVR): Use `<dtmf>` to navigate the IVR menu played by the main agent. `<dtmf>` can appear in any verbatim step.
> * **Outbound** (testing agent simulates an external IVR): Use `<ivr text="..." />` to play the IVR menu. `<ivr>` can appear in any verbatim step. When the scenario contains `<ivr>`, DTMF digits pressed by the main agent appear in the transcript — write your conditions using the specific digit (e.g., `"The main agent pressed 1"`).

### Example 3: Multi-Part Response with Follow-ups

```json theme={null}
{
  "role": "You are a customer with a complex request",
  "first_message": {
    "say": "Hi, I need help with my order",
    "then": [
      "<ai_generated>Also mention you haven't received a confirmation email</ai_generated>",
      "<ai_generated>And ask if you can get expedited shipping</ai_generated>"
    ]
  },
  "conditions": [
    {
      "when": "The agent responds to your requests",
      "say": "<ai_generated>Thank them and confirm the details</ai_generated>"
    }
  ]
}
```

**Turn-by-turn flow for Example 3:**

1. Testing agent sends the first message: "Hi, I need help with my order"
2. Main agent replies
3. The first `then` step fires — testing agent mentions the missing confirmation email
4. Main agent replies
5. The second `then` step fires — testing agent asks about expedited shipping
6. Main agent replies — the condition matches and fires

### Example 4: Using Advanced Tags

```json theme={null}
{
  "role": "You are a customer in a noisy environment",
  "first_message": "<background_noise sound=\"coffee-shop\" volume=\"0.3\">Hi, I'm calling about my order</background_noise>",
  "conditions": [
    {
      "when": "The agent asks you to repeat",
      "say": "<volume ratio=\"1.5\" />I said I'm calling about my order, <spell>ABC</spell> 123"
    },
    {
      "when": "The agent asks for more information",
      "say": "Let me check <silence time=\"2s\" /> here's the reference number"
    }
  ]
}
```

### Example 5: Live Data via a Custom Function

One `auto_run` function feeding both kinds of step: the first condition drops a fetched value into a **verbatim** step with a `{{function.*}}` placeholder, and its `then` step lets the testing agent phrase an **AI-generated** reply from the same response — no placeholder needed.

```json theme={null}
{
  "role": "You are a customer checking on an order",
  "functions": [
    {
      "name": "lookup",
      "type": "rest_api",
      "auto_run": true,
      "config": {
        "method": "GET",
        "url": "https://api.example.com/orders/{{test_profile.order_id}}",
        "timeout_seconds": 5,
        "response_mapping": {
          "status": { "path": "$.status", "default": "unknown" },
          "customer": { "path": "$.customer_name", "default": "unknown" }
        }
      }
    }
  ],
  "first_message": "Hi, I'd like to check on my order.",
  "conditions": [
    {
      "when": "The agent asks for the name on the order",
      "say": "It's under {{function.lookup.customer}}.",
      "then": [
        "<ai_generated>Ask whether the order status they see matches yours, mentioning the real status from the lookup.</ai_generated>"
      ]
    },
    {
      "when": "The agent confirms the order status",
      "say": "Great, that's all I needed. Thanks! <endcall />"
    }
  ]
}
```

**Turn-by-turn flow for Example 5:**

1. As the call connects, `lookup` runs once in the background (`auto_run`) — say it returns `{"status": "shipped", "customer_name": "Jane Doe"}`
2. Testing agent sends the first message: "Hi, I'd like to check on my order."
3. The agent asks for the name — the first condition's **verbatim** `say` fires with the placeholder substituted: "It's under Jane Doe."
4. The agent replies — the **AI-generated** `then` step fires; the testing agent is given the lookup results automatically and phrases them naturally: e.g. "On my side it shows as shipped — is that what you see?"
5. The agent confirms — the second condition closes the call

If the API call fails, the declared `default`s take over: the first condition speaks "It's under unknown." and the testing agent in the `then` step is told the lookup failed rather than inventing a status.

**On-demand variant (`auto_run: false`):** to run the lookup only at a specific point in the flow instead of at call start, turn off `auto_run` and invoke it with a tag — the function completes before the rest of the step is spoken:

```json theme={null}
{
  "role": "You are a customer checking on an order",
  "functions": [
    { "name": "lookup", "type": "rest_api", "auto_run": false,
      "config": { "method": "GET", "url": "https://api.example.com/orders/{{test_profile.order_id}}",
        "response_mapping": { "customer": { "path": "$.customer_name", "default": "unknown" } } } }
  ],
  "first_message": "Hi, I'd like to check on my order.",
  "conditions": [
    { "when": "The agent asks for the name on the order",
      "say": "One moment. <function name=\"lookup\"/> It's under {{function.lookup.customer}}." }
  ]
}
```

Nothing runs at call start; the request fires the moment that step executes, and the same verbatim / AI-generated rules apply from there.

## Best Practices

> [card] Clear Conditions
>
> Write `when` so it clearly describes **what the agent does** to trigger it.
>
> * ✅ "The agent asks for your email"
> * ❌ "Email"

> [card] Specific Steps
>
> Make steps specific and actionable.
>
> * ✅ "<ai_generated>Provide your email address</ai_generated>"
> * ❌ "<ai_generated>Answer the question</ai_generated>"

> [card] Logical Flow
>
> Order conditions in a logical conversation flow.
>
> Start with `first_message`, then follow natural conversation progression.

## Common Patterns

### Pattern 1: Conditional Branching

Handle different paths based on agent response:

```json theme={null}
{
  "role": "You are a customer requesting a refund",
  "first_message": "I would like to request a refund for my order",
  "conditions": [
    {
      "when": "The agent approves the refund",
      "say": "<ai_generated>Thank them and confirm the details</ai_generated>"
    },
    {
      "when": "The agent denies the refund",
      "say": "<ai_generated>Ask to speak with a manager</ai_generated>"
    }
  ]
}
```

### Pattern 2: Information Gathering

Progressively provide information as requested:

```json theme={null}
{
  "role": "You are a customer calling about your account",
  "first_message": "Hi, I'm calling about my account",
  "conditions": [
    {
      "when": "The agent asks for your name",
      "say": "<ai_generated>Provide your name</ai_generated>"
    },
    {
      "when": "The agent asks for your account number",
      "say": "<ai_generated>Provide your account number</ai_generated>"
    },
    {
      "when": "The agent asks for verification",
      "say": "<ai_generated>Provide your date of birth</ai_generated>"
    }
  ]
}
```

### Pattern 3: Scripted Sequence with `then`

Use `then` to deliver an exact sequence of messages across turns, with no conditions to match — each step fires automatically on the next turn after the main agent replies, regardless of what the main agent says:

```json theme={null}
{
  "role": "You are a customer updating your address",
  "first_message": {
    "say": "Hi, I need to update my address",
    "then": [
      "My new street is 123 Main Street",
      "City is Springfield",
      "Zip code is 62701"
    ]
  }
}
```

## Tips

> [!TIP]
>
> **Start Simple**: Begin with basic conditions and add complexity as needed. Test frequently.

> [!TIP]
>
> **Default to verbatim**: Exact wording makes runs reproducible. Reach for `<ai_generated>` only where the reply should vary.

> [!TIP]
>
> **Test Edge Cases**: Include conditions for unexpected agent responses or errors.

## Troubleshooting

> [accordion] Validation error: \<ai\_generated> is not allowed in the first message
>
> **Issue**: `first_message: the first message is always said verbatim; <ai_generated> is not allowed.`
>
> **Solution**: Write the opening line out word-for-word. If you want an improvised step right after it, put it in `first_message.then`:
>
> ```json theme={null}
> { "first_message": { "say": "Hi there", "then": ["<ai_generated>Explain why you are calling</ai_generated>"] } }
> ```

> [accordion] Validation error: \<ai\_generated> must wrap the whole step
>
> **Issue**: A step mixes verbatim text with an `<ai_generated>` part, or nests one inside another.
>
> **Solution**: A step is either entirely verbatim or entirely AI-generated. Split mixed content into two steps using `then`:
>
> ```json theme={null}
> { "when": "The agent asks about pricing", "say": "Sure.", "then": ["<ai_generated>Ask about discounts</ai_generated>"] }
> ```

> [accordion] Validation error: a tag in an \<ai\_generated> step
>
> **Issue**: For example, `conditions[0]: action contains <endcall> tag but the step is wrapped in <ai_generated>.`
>
> **Solution**: Tags only work in verbatim steps. Remove the `<ai_generated>` wrapper, or move the tag into its own verbatim `then` step.

> [accordion] Validation error: \<interruption> must be a then step
>
> **Issue**: `<interruption>` was used in `first_message` or a condition's `say`.
>
> **Solution**: Move it into a `then` step, at the very start of that step.

> [accordion] Validation error: scenario\_language is required
>
> **Issue**: Creating a Structured Tests evaluator fails with a language error.
>
> **Solution**: Assign a personality to the evaluator — the language is inferred automatically from the personality. Alternatively, set `scenario_language` explicitly in the request.

> [accordion] Condition Not Triggering
>
> **Issue**: A condition isn't triggering when expected.
>
> **Solutions**:
>
> * Make `when` more specific
> * Check if a previous condition is matching instead
> * Verify `when` describes what the agent does, not what the evaluator should do

> [accordion] A then step fires earlier or later than expected
>
> **Issue**: A `then` step fires at the wrong time.
>
> **Explanation**: Each `then` step fires on the **next turn** after the step before it — after the testing agent delivers that step and the main agent replies. It never fires in the same turn, and it does not wait for any particular reply.
>
> **Solutions**:
>
> * If a step should only fire after a specific reply, make it its own condition with a matching `when` instead of a `then` step
> * Keep `then` for messages that should follow regardless of what the agent says

> [accordion] Tags Not Working
>
> **Issue**: Special tags aren't producing the expected effect.
>
> **Solutions**:
>
> * Check tag syntax (spelling, attributes, closing format)
> * Check tag placement (some tags must be at the start or span the entire step)
> * Confirm the step is verbatim — tags don't work in `<ai_generated>` steps

> [accordion] First Message Not Sending
>
> **Issue**: The conversation doesn't start.
>
> **Solutions**:
>
> * Check that `first_message` is set — an empty `first_message` means the testing agent waits for the main agent to speak first
> * Check that the role is defined

> [accordion] Can't run — an attached audio clip is still transcribing / failed
>
> **Issue**: Runs are blocked with a message about an attached audio clip.
>
> **Solution**: A scenario can't run while any [attached audio](#attached-audio) clip is not `ready`. Wait for transcription to finish (`pending`/`processing`), or if a clip `failed`, re-upload it or remove it. Check clip statuses via `GET scenarios/{id}/condition-audio/`.

> [accordion] Attach audio option is missing on a step
>
> **Issue**: A step has no way to attach a recording.
>
> **Solution**: Audio attaches only to **verbatim** steps — uploading to an `<ai_generated>` step returns `Audio can only be attached to a verbatim step, not one wrapped in <ai_generated>`. Remove the `<ai_generated>` wrapper from the step first.

## Next Steps

> [card] [Test Profiles](https://docs.cekura.ai/documentation/key-concepts/evaluators/test-profile.md)
>
> Add identity information to your conditional actions evaluator

> [card] [Personalities](https://docs.cekura.ai/documentation/key-concepts/evaluators/personality.md)
>
> Configure language and behavioral characteristics

> [card] [Metrics](https://docs.cekura.ai/documentation/key-concepts/metrics/overview.md)
>
> Measure performance with custom metrics

> [card] [Prompting Guide](https://docs.cekura.ai/documentation/guides/prompting.md)
>
> Learn best practices for writing effective conditions


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.