> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cekura.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Update Evaluator
> Partially update an evaluator. Same field shape as create — see `scenarios_create` for details.
## OpenAPI
````yaml patch /test_framework/v1/scenarios/{id}/
openapi: 3.1.0
info:
title: Cekura API
version: v1
description: >-
Complete API documentation for the Cekura platform. This API provides
endpoints for testing, observing, and evaluating AI voice agents — including
managing agents, running evaluators, defining metrics, and analyzing call
quality.
servers:
- url: https://api.cekura.ai
security: []
paths:
/test_framework/v1/scenarios/{id}/:
patch:
tags:
- Scenarios
summary: Update an evaluator (scenario)
description: >-
Partially update an evaluator. Same field shape as create — see
`scenarios_create` for details.
**Tip:** to add or modify a single condition inside a
`conditional_actions` scenario, fetch the scenario with GET, mutate its
`conditional_actions` object, and PATCH the full object back. The server
replaces all conditions atomically.
operationId: scenarios-partial-update
parameters:
- in: path
name: id
schema:
type: integer
description: A unique integer value identifying this scenario.
required: true
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/PatchedSchemaPostScenario'
examples:
Rename+TweakInstructions:
value:
name: Refund flow v2
instructions: Ask for the order ID before refunding.
summary: Rename + tweak instructions
application/x-www-form-urlencoded:
schema:
$ref: '#/components/schemas/PatchedSchemaPostScenario'
multipart/form-data:
schema:
$ref: '#/components/schemas/PatchedSchemaPostScenario'
responses:
'200':
content:
application/json:
schema:
$ref: '#/components/schemas/SchemaScenario'
description: ''
'400':
content:
application/json:
schema:
type: object
properties:
field_name:
type: array
items:
type: string
description: ''
security:
- api_key: []
components:
schemas:
PatchedSchemaPostScenario:
type: object
description: ''
properties:
agent:
type:
- integer
- 'null'
description: ID of the agent. Either agent or project must be provided.
project:
type:
- integer
- 'null'
description: ID of the project. Either agent or project must be provided.
assistant_id:
type: string
writeOnly: true
description: |
Alternative to agent ID - the assistant ID to use for this scenario
Example: `"asst_1234567890"`
name:
type: string
description: Name of the evaluator
personality:
type: integer
description: The personality of the evaluator
scenario_type:
enum:
- instruction
- conditional_actions
type: string
x-spec-enum-id: 97f8e4b948b4e205
default: instruction
description: >-
Type of scenario to create:
- `instruction`: Standard instruction-based scenario where the
instructions field contains plain text
- `conditional_actions`: Conditional action-based scenario where the
instructions field contains a JSON structure
Default: `instruction`
* `instruction` - Instruction
* `conditional_actions` - Conditional Actions
instructions:
type: string
description: >
Scenario Instructions - the format depends on the scenario_type:
**For scenario_type="instruction":**
Provide plain text instructions that describe the scenario behavior.
Example: `"You are a customer calling to inquire about your order
status. Be polite and provide order number 12345 when asked."`
**For scenario_type="conditional_actions":**
Use the structured `conditional_actions` field instead. If you send
via `instructions`, pass a JSON-formatted string with the following
structure:
```json
{
"role": "A friendly customer calling about an appointment",
"conditions": [
{
"id": 0,
"condition": "FIRST_MESSAGE",
"action": "Hi, I'd like to check on my upcoming appointment ",
"type": "standard",
"fixed_message": true
},
{
"id": 1,
"condition": "The agent asks for your name",
"action": "My name is Sarah Johnson",
"type": "standard",
"fixed_message": true
},
{
"id": 2,
"condition": "The agent asks for your date of birth",
"action": "January first, nineteen ninety",
"type": "standard",
"fixed_message": true
},
{
"id": 3,
"condition": "The agent confirms your identity and provides appointment details",
"action": "Thank you, that's all I needed ",
"type": "standard",
"fixed_message": true
},
{
"id": 4,
"condition": 3,
"action": "",
"type": "action_followup",
"fixed_message": true
}
]
}
```
**Note:** When sent via `instructions`, the value must be a
JSON-formatted string (e.g. `JSON.stringify(...)` /
`json.dumps(...)`). For new integrations, use the structured
`conditional_actions` field below.
**Structure Details:**
- **`role`** (string): Personality/role description for the scenario
actor
- **`conditions`** (array): Ordered list of conditional actions that
define the conversation flow
**Each condition object has:**
- **`id`** (integer): Unique sequential identifier (0, 1, 2, 3...)
- **`condition`** (string | integer): The trigger that activates this action
- `"FIRST_MESSAGE"` — Triggers at conversation start (use for `id: 0`)
- Natural-language description of the main agent's observable action, written from a third-person observer perspective (e.g. `"The agent asks for your date of birth"`, `"The agent confirms the cancellation"`). Matched semantically against the conversation context.
- Integer (e.g. `3`) — References another condition's `id` for sequential `action_followup` chaining
- **`action`** (string): What the scenario actor says or does
- **SSML pause tags:** `` (interruptible) or `` (not interruptible). `` lets the main agent jump in mid-pause; `` enforces dead air.
- **Emotion markers:** `[laughter]`, `[sigh]`, etc. (can repeat like `[laughter] [laughter]`)
- **Special actions:** `` to end the call
- **`type`** (string): Either `"standard"` (normal response) or `"action_followup"` (fires on the testing agent's next turn after the referenced condition — one main-agent reply elapses in between, regardless of its content; never fires in the same turn as its parent)
- **`fixed_message`** (boolean): Set to `true` for fixed responses, `false` for dynamic
**Usage Tips:**
- Always start with `id: 0` and `condition: "FIRST_MESSAGE"`
- Write conditions as third-person descriptions of what the main
agent does (e.g. `"The agent confirms the cancellation"`)
- Use the last condition to trigger `` for clean call
termination
- Combine SSML tags and emotion markers naturally: `"Great [laughter] [laughter]"`
conditional_actions:
allOf:
- $ref: '#/components/schemas/ConditionalActions'
description: >-
Structured conditional-actions object for
`scenario_type="conditional_actions"`. Use either
`conditional_actions` (structured) or `instructions` (plain text) —
not both.
expected_outcome_prompt:
type: string
description: |
Expected Outcome Prompt
Example: `"The user should be able to complete the order"`
metrics:
type: array
items:
type: integer
description: |
List of metric IDs to associate with this evaluator
Example: `[123, 456, 789]`
tags:
type: array
items:
type: string
description: |
List of tags to associate with the evaluator
Example: `["tag1", "tag2", "tag3"]`
tool_ids:
type: array
items:
type: string
description: List of tool IDs to use for evaluator
example:
- TOOL_DTMF
- TOOL_END_CALL
- TOOL_END_CALL_ONLY_ON_TRANSFER
test_profile:
type: integer
description: The test profile ID to use for the evaluator
phone_number:
type: integer
description: >-
The phone number ID to use for the evaluator (works for both inbound
and outbound)
inbound_phone_number:
type: integer
description: >-
(Deprecated) The inbound phone number ID to use for the evaluator.
Use phone_number instead, which works for both inbound and outbound.
folder_path:
type:
- string
- 'null'
description: >-
Dot-separated path of the folder to assign this evaluator to (e.g.
"Sales.Inbound"). Folders must be created before use — they are not
auto-created. Use `scenarios_create_folder_create` to create each
level before assigning evaluators. Example: to use "Sales.Inbound",
first create "Sales", then create "Inbound" inside it.
SchemaScenario:
type: object
description: ''
properties:
id:
type: integer
readOnly: true
name:
type: string
description: Name of the evaluator
agent:
type: integer
description: |
Agent ID
Example: `2142`
project:
type:
- integer
- 'null'
assistant_id:
type: string
writeOnly: true
description: |
Alternative to agent ID - the assistant ID to use for this scenario
Example: `"asst_1234567890"`
personality:
type: integer
description: ID of the personality
personality_name:
type: string
description: Name of the personality
agent_name:
type: string
readOnly: true
description: |
Name of the agent associated with this scenario
Example: `"Customer Support Agent"`
tags:
type: array
items:
type: string
description: |
List of tags of the evaluators
Example: `["tag1", "tag2", "tag3"]`
tool_ids:
description: |
List of tool IDs to associate with this scenario
Example: `["TOOL_DTMF", "TOOL_END_CALL"]`
metrics:
type: array
items:
type: integer
description: |
List of metric IDs
Example: `[123, 456, 789]`
metric_names:
type: array
items:
type: string
description: |
List of metric names
Example: `["Metric 1", "Metric 2", "Metric 3"]`
phone_number:
type: string
description: 'Phone number eg: +17776666333'
outbound_phone_number_data:
allOf:
- $ref: '#/components/schemas/PhoneNumberInbound'
readOnly: true
description: |
Phone number used for outbound calls in this scenario
Example: `"+1234567890"`
first_message:
type: string
description: First message of the evaluator
inbound_phone_number:
type:
- integer
- 'null'
inbound_phone_number_data:
allOf:
- $ref: '#/components/schemas/PhoneNumberInbound'
readOnly: true
description: >
(Deprecated) Phone number used for inbound calls in this scenario.
Use outbound_phone_number_data instead.
After CEK-6517, both phone_number and inbound_phone_number fields
are synced, so outbound_phone_number_data contains the unified phone
number for both inbound and outbound calls.
Example: `{"id": 123, "number": "+1234567890", "phone_number_id":
"abc123"}`
instructions:
type: string
description: >
Scenario Instructions - format depends on scenario_type:
- For `instruction` type: Plain text instructions
- For `conditional_actions` type: JSON string with role and
conditions array. Each condition has id, condition trigger
(FIRST_MESSAGE, a natural-language third-person description of the
main agent's observable action — e.g. `"The agent asks for your date
of birth"`, matched semantically — or an integer reference to
another condition's id), action (text with optional `` (interruptible) or `` (not
interruptible) and `[laughter]` tags), type
(standard/action_followup), and fixed_message flag
conditional_actions:
writeOnly: true
description: >-
Structured conditional-actions object for
`scenario_type="conditional_actions"`. Use either
`conditional_actions` (structured) or `instructions` (plain text) —
not both.
Example:
```json
{
"role": "You are a patient calling to schedule an appointment",
"conditions": [
{"id": 0, "condition": "FIRST_MESSAGE", "action": "Hi, I need to schedule an appointment",
"type": "standard", "fixed_message": true},
{"id": 1, "condition": "when the agent asks for your name",
"action": "Provide your full name: John Smith", "type": "standard", "fixed_message": false}
]
}
```
simulation_description:
type: string
description: Simulation Description
information_fields:
type: object
additionalProperties: {}
description: |
Information fields of the evaluator
Example:
```json
{
"user_name": "John Doe",
"user_email": "john.doe@example.com",
}
```
expected_outcome_prompt:
type: string
description: |
Expected outcome prompt of the evaluator
Example: `"The user should be able to complete the order"`
scenario_language:
enum:
- af
- ar
- bn
- bg
- zh
- cs
- da
- nl
- en
- et
- fi
- fr
- de
- el
- gu
- hi
- he
- hu
- id
- it
- ja
- kn
- ko
- ms
- ml
- mr
- multi
- 'no'
- pl
- pa
- pt
- ro
- ru
- sk
- es
- sv
- th
- tr
- tl
- ta
- te
- uk
- vi
type: string
x-spec-enum-id: b822ad429c7b7fec
description: |-
Language of the scenario
* `af` - Afrikaans
* `ar` - Arabic
* `bn` - Bengali
* `bg` - Bulgarian
* `zh` - Chinese Simplified
* `cs` - Czech
* `da` - Danish
* `nl` - Dutch
* `en` - English
* `et` - Estonian
* `fi` - Finnish
* `fr` - French
* `de` - German
* `el` - Greek
* `gu` - Gujarati
* `hi` - Hindi
* `he` - Hebrew
* `hu` - Hungarian
* `id` - Indonesian
* `it` - Italian
* `ja` - Japanese
* `kn` - Kannada
* `ko` - Korean
* `ms` - Malay
* `ml` - Malayalam
* `mr` - Marathi
* `multi` - Multilingual
* `no` - Norwegian
* `pl` - Polish
* `pa` - Punjabi
* `pt` - Portuguese
* `ro` - Romanian
* `ru` - Russian
* `sk` - Slovak
* `es` - Spanish
* `sv` - Swedish
* `th` - Thai
* `tr` - Turkish
* `tl` - Tagalog
* `ta` - Tamil
* `te` - Telugu
* `uk` - Ukrainian
* `vi` - Vietnamese
scenario_type:
type: string
description: >
Type of scenario:
- `instruction`: Standard instruction-based scenario (plain text
instructions)
- `conditional_actions`: Conditional action-based scenario (JSON
with role, conditions, SSML tags, emotion markers)
- `real_world_smart`: Real world smart scenario
- `real_world_fixed`: Real world fixed scenario
suite_type:
enum:
- undefined
- infrastructure
type: string
x-spec-enum-id: 643f6a8cdc799a77
description: |-
Type of predefined suite (e.g., infrastructure)
* `undefined` - Undefined
* `infrastructure` - Infrastructure
is_predefined:
type: boolean
description: Whether this is a predefined scenario (template)
test_profile:
type:
- integer
- 'null'
test_profile_data:
allOf:
- $ref: '#/components/schemas/AIAgentTestProfile'
readOnly: true
description: >
Details of the test profile associated with this scenario.
`information` uses the
sectioned shape: `main_agent_variables` are sent to the agent under
test as
dynamic variables at call time; `testing_agent_variables` are
persona/context
used by Cekura's simulated caller. Legacy flat dicts (no section
keys) still
work — at call time they are delivered to both sides for backward
compatibility.
Example:
```json
{
"id": "",
"agent_id": "",
"name": "",
"information": {
"main_agent_variables": {
"user_id": "",
"account_id": ""
},
"testing_agent_variables": {
"user_name": "",
"user_email": "",
"order_id": ""
}
}
}
```
dynamic_variable_values:
readOnly: true
description: >-
REMOVED FROM API. Reads return the historical value for legacy rows
(`null` for new rows). Writes are no longer accepted — supplying a
non-null value yields a 400 error. Put dynamic-variable values in
`test_profile_data.information.main_agent_variables` instead: create
the test profile via POST /test-profiles/ and attach it via the
`test_profile` field on the scenario.
generated_mock_tool_entries:
oneOf:
- {}
- type: 'null'
description: >-
Expected mock tool calls for this evaluator. Each entry has shape
{tool_id, tool_name, new_entry: {input, output}}. Auto-generated
during scenario generation when the agent has mock tools attached;
can also be set or edited manually.
folder_path:
type:
- string
- 'null'
- 'null'
description: >-
Dot-separated path of the folder to assign this evaluator to (e.g.
"Sales.Inbound"). Folders must be created before use — they are not
auto-created. Use `scenarios_create_folder_create` to create each
level before assigning evaluators. Example: to use "Sales.Inbound",
first create "Sales", then create "Inbound" inside it.
max_duration:
type:
- integer
- 'null'
description: >
Max call duration in seconds for this scenario. If not provided or
set to null, the scenario will use the project's max_call_duration
setting.
Valid range: 10-3600 seconds (10 seconds to 60 minutes)
Example: `600` (10 minutes)
ConditionalActions:
type: object
description: >-
Structured conditional-actions object for
`scenario_type="conditional_actions"`.
Use either `conditional_actions` (structured) or `instructions` (plain
text)
on a scenario — not both.
properties:
role:
type: string
description: >-
Caller role / persona for this scenario. Example: `"You are a
frustrated patient calling about a billing error"`.
conditions:
type: array
items:
$ref: '#/components/schemas/Condition'
description: Ordered list of conditions and their actions.
required:
- conditions
- role
PhoneNumberInbound:
type: object
properties:
id:
type: integer
readOnly: true
number:
type: string
maxLength: 15
phone_number_id:
type: string
maxLength: 255
required:
- number
- phone_number_id
AIAgentTestProfile:
type: object
description: ''
properties:
id:
type: integer
readOnly: true
project:
type:
- integer
- 'null'
agent:
type:
- integer
- 'null'
agent_name:
type: string
readOnly: true
description: |
Name of the agent associated with this test profile
Example: `"Customer Support Agent"`
name:
type: string
maxLength: 255
information:
description: >
Variables for the test profile, split by which agent receives them.
```json
{
"main_agent_variables": {"user_id": "U-123"},
"testing_agent_variables": {"user_name": "John Doe", "user_email": "john.doe@example.com"}
}
```
`main_agent_variables` are sent to the agent under test as dynamic
variables. `testing_agent_variables` shape the simulated caller's
persona. A legacy flat dict (no section keys) is accepted for
backward compatibility and is sent to both agents.
Keys in `main_agent_variables` starting with `X-` are additionally
sent as custom SIP headers (SIP runs) or connection headers
(WebSocket runs) to the agent under test. For SIP runs this is the
only way to pass custom headers; `X-Run-Id`, `X-Scenario-Id`, and
`X-Result-Id` are reserved and always set by Cekura.
created_by:
type: integer
readOnly: true
description: ID of the user who created this test profile
last_updated_by:
type: integer
readOnly: true
description: ID of the user who last updated this test profile
created_at:
type: string
format: date-time
readOnly: true
description: Timestamp when the test profile was created
updated_at:
type: string
format: date-time
readOnly: true
description: Timestamp when the test profile was last updated
required:
- name
Condition:
type: object
description: One entry in a ConditionalActions flow.
properties:
id:
type: integer
description: Sequential identifier. Start at 0 for the first condition.
condition:
type: string
description: >-
Trigger expression. Special values:
- `"FIRST_MESSAGE"` — fires at conversation start (use for `id: 0`)
- `"contains \"\""` — fires when the agent utterance contains
the substring
- `"contains \"\" OR contains \"\""` — OR-combined substrings
- `"contains \"\" AND contains \"\""` — AND-combined
substrings
- Integer (e.g. `2`) — references another condition id for
sequential follow-up
action:
type: string
description: >-
What the simulated caller says or does. Supports:
- SSML pauses: `` (interruptible) or `` (not interruptible)
- Emotion markers: `[laughter]`, `[sigh]`
- Call control: ``
type:
enum:
- standard
- action_followup
type: string
x-spec-enum-id: adcfc876f3a8758f
default: standard
description: >-
`standard` — normal response to the matched condition.
`action_followup` — fires on the testing agent's next turn after the
referenced condition was executed (one main-agent reply elapses in
between). Firing is deterministic and does not depend on the reply's
content; it never fires in the same turn as its parent.
* `standard` - standard
* `action_followup` - action_followup
fixed_message:
type: boolean
default: false
description: >-
If `true`, deliver `action` verbatim (no LLM paraphrasing). Use for
exact-phrasing tests.
required:
- action
- condition
- id
securitySchemes:
api_key:
type: apiKey
name: X-CEKURA-API-KEY
in: header
description: >-
API Key Authentication. It should be included in the header of each
request.
````