Skip to main content
POST
Run evaluators against an agent over any transport

Authorizations

X-CEKURA-API-KEY
string
header
required

API Key Authentication. It should be included in the header of each request.

Body

transport
enum<string>
required

How the simulated user reaches the agent. Must be enabled on the agent; aiagents_retrieve lists the options under enabled_connections. websocket and pipecat_v1 carry their connection details per scenario instead.

  • voice - Voice
  • sip - SIP
  • text - Text
  • email - Email
  • vapi_webrtc - VAPI WebRTC
  • retell_webrtc - Retell WebRTC
  • elevenlabs - ElevenLabs
  • livekit_v2 - LiveKit
  • pipecat_v2 - Pipecat Cloud
  • chirp - Raw-PCM WebSocket
  • agora - Agora
  • websocket - WebSocket (per-scenario URLs)
  • pipecat_v1 - Pipecat (per-scenario room URLs)
Available options:
voice,
sip,
text,
email,
vapi_webrtc,
retell_webrtc,
elevenlabs,
livekit_v2,
pipecat_v2,
chirp,
agora,
websocket,
pipecat_v1
agent
integer

Agent ID to run against. Required unless every selected evaluator already belongs to an agent.

scenarios
integer[]

Evaluator IDs to run, in order. Provide exactly one of scenarios, tags, folder_path. The dashboard may also send its selection shorthand ("all", a count, or a folder path).

tags
string[]

Run every evaluator carrying any of these tags.

folder_path
string

Dot-separated folder path whose evaluators run. Requires project_id.

project_id
integer

Project ID. Required with folder_path.

frequency
integer
default:1

Times each evaluator runs. Total runs = evaluators × frequency × personalities × test profiles.

Required range: x >= 1
name
string

Label for the result.

Maximum string length: 255
personality_ids
integer[] | null

Personality IDs to run each evaluator with instead of its own. Only personalities enabled for the project.

test_profile_ids
integer[] | null

Test profile IDs to run each evaluator with instead of its own.

concurrency_limit
integer | null

Cap on runs executing in parallel. Default: unbounded.

Required range: x >= 1
calls_per_second
number<double> | null

Most calls started per second for this run, a number from 0.1 to 4 (e.g. 0.5 starts one call every 2 seconds). Leave unset for no limit.

Required range: 0.1 <= x <= 4
mock_tool_names
string[] | null

Mock tools to activate for this run. Omit to mock every configured tool; send an empty list to mock none.

agent_version_id
string

Version label or numeric history id of a stored agent version to run against instead of the live agent.

outbound_phone_number
string

voice, text: override the number dialed, in E.164 format.

agent_number
string

voice, text: override the number the agent under test receives calls on, in E.164 format.

mode
enum<string>

voice: use one phone number for every run (same_number) or rotate numbers (different_numbers).

  • same_number - same_number
  • different_numbers - different_numbers
Available options:
same_number,
different_numbers
sip_endpoint
string

sip: override the agent's SIP endpoint for this run.

websocket_url
string

text: websocket URL the simulated user connects to instead of the agent's chat gateway. chirp: override the agent's raw-PCM websocket endpoint.

Pattern: ^wss?://
websocket_headers
object

text: headers sent on the websocket connection, merged over the agent's configured headers.

agent_email
string<email>

email: override the agent's configured email address for this run.

vapi_assistant_id
string

vapi_webrtc: VAPI assistant ID to test. Defaults to the agent's stored assistant id.

retell_agent_id
string

retell_webrtc: Retell agent ID to test. Defaults to the agent's stored assistant id.

elevenlabs_agent_id
string

elevenlabs: ElevenLabs agent ID to test. Defaults to the agent's stored assistant id.

elevenlabs_data
object

elevenlabs: per-run overrides (elevenlabs_base_url_override, elevenlabs_environment, elevenlabs_branch_id).

livekit_data
object

livekit_v2: per-run overrides (agent_name, url, config). Credentials come from the agent or project.

pipecat_data
object

pipecat_v2: per-run overrides (pipecat_agent_name, config, room_properties). Credentials come from the agent or project.

websocket_urls
object[]

websocket: the evaluators to run, each with its own websocket URL. Replaces scenarios.

pipecat_rooms
object[]

pipecat_v1: the evaluators to run, each with its own Daily.co room URL. Replaces scenarios.

Response

name
string
required

Name of the Result Example: "Test Result 1"

Maximum string length: 255
id
integer
read-only

Unique identifier for the Result Example: 123

agent
integer
read-only

ID of the AI agent that was tested Example: 123

agent_inbound
boolean
read-only

Whether the agent is configured for inbound calls (true) or outbound calls (false) Example: true or false

status
enum<string>
default:pending

Current status of the result

  • running - Running
  • completed - Completed
  • failed - Failed
  • pending - Pending
  • in_progress - In Progress
  • evaluating - Evaluating
  • in_queue - In Queue
  • timeout - Timeout
  • cancelled - Cancelled
  • scaling_up - Scaling Up
Available options:
running,
completed,
failed,
pending,
in_progress,
evaluating,
in_queue,
timeout,
cancelled,
scaling_up
success_rate
number<double>
default:0

Percentage of runs that passed, 0 to 100

run_as_text
boolean
default:false

Whether this test was run in text mode instead of voice mode Example: true or false

is_cronjob
string
read-only

Whether this result was created by a scheduled cronjob Example: true or false

runs
object
read-only

Run summaries for this result. For full run data including transcripts and metric scores, fetch the result detail endpoint — there runs is a dict keyed by run ID.

created_at
string<date-time>
read-only

Timestamp when this test result was created Example: 2021-01-01 00:00:00

updated_at
string<date-time>
read-only

Timestamp when this test result was last updated