Run Evaluator (API Chat)
Start API chat runs you drive turn-by-turn over REST. No chat provider needs to be configured.
Authorizations
API Key Authentication. It should be included in the header of each request.
Body
Schema for the run_scenarios_api_chat endpoint (customer-driven pull mode).
Same selection fields as the other run endpoints (agent_id, scenarios/tags/
folder_path, frequency, personality/test-profile overrides) plus timeout.
ID of the agent to run evaluators for
Alternative to agent ID - the assistant ID to use for this scenario
Example: "asst_1234567890"
Label text for result
List of evaluator IDs to run. Either scenarios, tags, or folder_path must be provided.
Example: [11, 22, 33]
To run evaluators by tag or folder, use the tags or folder_path fields instead.
List of tags to filter evaluators to run. Either evaluators or tags must be provided.
Example: ["tag1", "tag2", "tag3"]
Dot-separated folder path to select evaluators from (e.g. "Sales.Inbound").
Mutually exclusive with scenarios and tags. Requires project_id.
Project ID. Required when using folder_path (or when passing a folder path string via scenarios).
Example: 1376
The number of times each evaluator will run
Example: 1
x >= 1Personality IDs to override for this run. Defaults to each scenario's personality.
Test-profile IDs to override for this run. Defaults to each scenario's test profile.
Overall run timeout in seconds — the whole conversation's max lifetime (NOT a per-message timeout). When it elapses the run is finalized and scored automatically. Defaults to the project's configured duration. Capped at 3600 (1 hour).
1 <= x <= 3600Response
Name of the Result
Example: "Test Result 1"
255Unique identifier for the Result
Example: 123
ID of the AI agent that was tested
Example: 123
Whether the agent is configured for inbound calls (true) or outbound calls (false)
Example: true or false
Current status of the result
running- Runningcompleted- Completedfailed- Failedpending- Pendingin_progress- In Progressevaluating- Evaluatingin_queue- In Queuetimeout- Timeoutcancelled- Cancelledscaling_up- Scaling Up
running, completed, failed, pending, in_progress, evaluating, in_queue, timeout, cancelled, scaling_up Success rate of the test runs as a decimal (0.0 to 1.0)
Whether this test was run in text mode instead of voice mode
Example: true or false
Whether this result was created by a scheduled cronjob
Example: true or false
Run summaries for this result. For full run data including transcripts and metric scores, fetch the result detail endpoint — there runs is a dict keyed by run ID.
Timestamp when this test result was created
Example: 2021-01-01 00:00:00
Timestamp when this test result was last updated