> ## Documentation Index > Fetch the complete documentation index at: https://docs.cekura.ai/llms.txt > Use this file to discover all available pages before exploring further. # Run Evaluator Voice > Execute evaluators against your agent using a real voice / phone / SIP call. This is the primary 'run your evaluators' endpoint for voice-mode testing. ## OpenAPI ````yaml post /test_framework/v1/scenarios/run_scenarios/ openapi: 3.1.0 info: title: Cekura API version: v1 description: >- Complete API documentation for the Cekura platform. This API provides endpoints for testing, observing, and evaluating AI voice agents — including managing agents, running evaluators, defining metrics, and analyzing call quality. servers: - url: https://api.cekura.ai security: [] paths: /test_framework/v1/scenarios/run_scenarios/: post: tags: - Evaluators summary: Run evaluators over a voice call description: >- VOICE RUN: Execute evaluators against your agent using a real voice / phone / SIP call. This is the primary 'run your evaluators' endpoint for voice-mode testing. Requires `telephony.phone_number` configured on the agent (any provider). **Run-count math:** total calls = `len(scenarios) × frequency × max(1, len(personality_ids)) × max(1, len(test_profile_ids))`. Each call is billed at the voice-simulation rate — check `frequency` carefully. **Concurrency:** use `concurrency_limit` to cap parallel calls. Default: unbounded (subject to provider rate limits). **Scenario selection (pick one):** - `scenarios` — list of integer evaluator IDs (e.g. `[101, 102]`) - `tags` — select by tag - `folder_path` — select by folder path (e.g. `"Sales.Inbound"`); requires `project_id` **Phone-number overrides:** - `outbound_phone_number` — override the number dialed - `agent_number` — override the contact number the inbound agent receives calls on - `mode` — `same_number` (all runs use one pair) vs `different_numbers` **When to use this vs others** (pick the endpoint matching the agent's configured connection — check `aiagents_retrieve`): - `run_scenarios_text` — text / SMS / chat mode; cheaper, no voice cost - `run_scenarios_with_websockets` — each scenario carries its own websocket URL - `run_scenarios_pipecat` — Pipecat/Daily.co WebRTC deployment **Personality override:** `personality_ids` must be IDs that are already enabled for the agent's project. Check which are enabled via `aiagents_retrieve` → `enabled_personalities` field. To enable a new personality, call `aiagents_partial_update` with the updated `enabled_personalities` list before using it here. Note: enabling a personality applies project-wide, not just to one agent. Passing an ID that is not enabled returns `["Invalid personalities provided."]`. **Scheduled runs:** to run on a recurring cron schedule, create a CronJob via `POST /test_framework/v1/cron_jobs/` with `execution_mode="voice"` instead of looping this endpoint. **Polling for completion:** the response contains a `result.id` with `status: 'running'`. Poll `GET /test_framework/v1/results/{result_id}/` until `status` is 'completed' or 'failed'. There is no separate progress endpoint for runs. Check `failed_reasons` for infrastructure errors; check `runs[run_id].error_message` for per-run errors. **Outbound dial window:** for outbound runs (agent `inbound=false`), your agent must dial the run's `outbound_number` before `outbound_dial_window_closes_at` (default 300s after the run reaches `waiting_for_call` / pending state). Dials received after that deadline are dropped and the run returns `status: timeout`. operationId: scenarios-run-voice_2 requestBody: content: application/json: schema: $ref: '#/components/schemas/SchemaPostRunScenarios' examples: RunSelectedScenarios(simplest): value: agent_id: 12 scenarios: - 101 - 102 - 103 frequency: 1 summary: Run selected scenarios (simplest) RunAnEntireFolder,3×Each,5InParallel: value: agent_id: 12 folder_path: Sales.Inbound project_id: 1376 frequency: 3 concurrency_limit: 5 name: Nightly Sales.Inbound regression summary: Run an entire folder, 3× each, 5 in parallel OverridePersonalities+TestProfiles(multipliesRunCount): value: agent_id: 12 scenarios: - 101 frequency: 2 personality_ids: - 3 - 7 test_profile_ids: - 20 - 21 summary: Override personalities + test profiles (multiplies run count) application/x-www-form-urlencoded: schema: $ref: '#/components/schemas/SchemaPostRunScenarios' multipart/form-data: schema: $ref: '#/components/schemas/SchemaPostRunScenarios' responses: '201': content: application/json: schema: type: object properties: id: type: integer description: ID of the result agent: type: integer description: ID of the agent status: type: string enum: - pending - running - completed - failed description: Status of the result run_as_text: type: boolean description: Whether the scenario ran as text or not example: false runs: type: array items: type: object properties: id: type: integer description: ID of the run status: type: string enum: - pending - running - completed - failed description: Status of the run scenario: type: integer description: ID of the scenario number: type: string description: >- For outbound runs (agent.inbound=False). The given number that must be called from the phone number configured in the cekura agent. example: '+11234567890' inbound_number: type: - string - 'null' description: >- For inbound runs (agent.inbound=True). The agent's configured phone number will receive calls from this number. example: '+11234567890' scenario_name: type: string description: Name of the scenario test_profile_data: type: - object - 'null' description: >- Details of the test profile associated with this run scenario outbound_dial_window_opens_at: type: - string - 'null' format: date-time description: >- When the outbound dial window opened (run transitioned to pending/waiting-for-call). Null until dispatched. outbound_dial_window_seconds: type: - integer - 'null' description: Duration of the outbound dial window in seconds. outbound_dial_window_closes_at: type: - string - 'null' format: date-time description: >- Deadline by which your agent must dial the outbound number (opens_at + outbound_dial_window_seconds). Dials after this are dropped and the run returns status: timeout. examples: - id: 274 status: pending scenario: 1 number: null inbound_number: '+11234567890' scenario_name: Customer Support Call (Agent Inbound = True) test_profile_data: null outbound_dial_window_opens_at: null outbound_dial_window_seconds: null outbound_dial_window_closes_at: null - id: 275 status: pending scenario: 2 number: '+11234567890' inbound_number: null scenario_name: Outbound Sales Call (Agent Inbound = False) test_profile_data: null outbound_dial_window_opens_at: '2026-06-24T10:47:30Z' outbound_dial_window_seconds: 300 outbound_dial_window_closes_at: '2026-06-24T10:52:30Z' created_at: type: string format: date-time example: '2025-02-25T21:00:01.990052Z' examples: InboundScenario: value: id: 167 agent: 1 status: pending success_rate: 0 run_as_text: false runs: - id: 274 scenario: 1 number: null inbound_number: '+11234567890' outbound_dial_window_opens_at: null outbound_dial_window_seconds: null outbound_dial_window_closes_at: null - id: 273 scenario: 2 number: null inbound_number: '+11234567891' outbound_dial_window_opens_at: null outbound_dial_window_seconds: null outbound_dial_window_closes_at: null created_at: '2025-02-25T21:00:01.990052Z' summary: Inbound Scenario OutboundScenario: value: id: 167 agent: 1 status: pending success_rate: 0 run_as_text: false runs: - id: 274 scenario: 1 number: '+11234567890' inbound_number: null outbound_dial_window_opens_at: '2026-06-24T10:47:30Z' outbound_dial_window_seconds: 300 outbound_dial_window_closes_at: '2026-06-24T10:52:30Z' - id: 273 scenario: 2 number: '+11234567891' inbound_number: null outbound_dial_window_opens_at: '2026-06-24T10:47:30Z' outbound_dial_window_seconds: 300 outbound_dial_window_closes_at: '2026-06-24T10:52:30Z' created_at: '2025-02-25T21:00:01.990052Z' summary: Outbound Scenario description: '' '400': content: application/json: schema: type: object properties: field_name: type: array items: type: string description: '' security: - api_key: [] components: schemas: SchemaPostRunScenarios: type: object description: '' properties: agent_id: type: integer description: ID of the agent to run evaluators for assistant_id: type: string writeOnly: true description: | Alternative to agent ID - the assistant ID to use for this scenario Example: `"asst_1234567890"` scenarios: type: array items: type: integer description: > List of evaluator IDs to run. Either scenarios, tags, or folder_path must be provided. Example: `[11, 22, 33]` To run evaluators by tag or folder, use the `tags` or `folder_path` fields instead. tags: type: array items: type: string description: > List of tags to filter evaluators to run. Either evaluators or tags must be provided. Example: `["tag1", "tag2", "tag3"]` folder_path: type: string description: > Dot-separated folder path to select evaluators from (e.g. `"Sales.Inbound"`). Mutually exclusive with `scenarios` and `tags`. Requires `project_id`. project_id: type: integer description: > Project ID. Required when using `folder_path` (or when passing a folder path string via `scenarios`). Example: `1376` frequency: type: integer minimum: 1 default: 1 description: | The number of times each evaluator will run Example: `1` name: type: string description: Label text for result outbound_phone_number: type: string description: | Override the phone number to use for outbound calls. Example: "+1234567890" personality_ids: type: - array - 'null' items: type: integer description: > List of personality IDs to override for this run. If not provided, uses the scenario's default personality. Example: `[123, 456, 789]` test_profile_ids: type: - array - 'null' items: type: integer description: > List of test profile IDs to override for this run. If not provided, uses the scenario's default test profile. Example: `[123, 456, 789]` personality_id: type: - integer - 'null' description: >- Single personality ID override. Merged into personality_ids; prefer personality_ids for multiple. test_profile_id: type: - integer - 'null' description: >- Single test profile ID override. Merged into test_profile_ids; prefer test_profile_ids for multiple. mode: enum: - same_number - different_numbers type: string x-spec-enum-id: 9ea0c9743bd9d873 default: same_number description: |- Using same or different phone numbers for each evaluation * `same_number` - same_number * `different_numbers` - different_numbers agent_number: type: string description: > Override the phone number from which the agent under test receives the call during outbound runs. This allows overriding the default agent contact number for testing purposes. Example: "+1234567890" concurrency_limit: type: integer minimum: 1 description: > Cap on the number of evaluator runs executed in parallel. Default: unbounded (subject to your provider's rate limits). Use this to avoid hitting provider rate limits on large frequency or folder runs. Example: `5` mock_tool_names: type: array items: type: string description: > Names of mock tools to activate for this run. Cekura swaps those tools' URLs to mock endpoints while the run is active and restores the originals when it finishes. Omit (or send null) to mock every tool configured on the agent; send an empty list to mock none so every tool keeps its original endpoint. Example: `["get_appointments", "cancel_booking"]` securitySchemes: api_key: type: apiKey name: X-CEKURA-API-KEY in: header description: >- API Key Authentication. It should be included in the header of each request. ````