Runs & Results

Setup steps and authentication are in the Overview. This page covers triggering runs and reading their output.

A run is one execution of a scenario against an agent. A result is the parent batch (one trigger → many runs, one per scenario × personality combination). The CLI and SDK expose both.

Pick a mode

The right command depends on the transport:

Mode	When to use
`text`	Fast functional check — no audio, scenarios run as text exchanges
`voice`	Outbound voice call via your provider (Vapi/Retell/etc.)
`chirp`	Cekura’s hosted voice runner
`livekit_v2` / `pipecat_v2`	Self-hosted LiveKit or Pipecat agents
`sip`	SIP-based provider (e.g. Plivo, Twilio SIP)
`websocket`	Custom WebSocket protocol
`vapi_webrtc`, `retell_webrtc`, `elevenlabs`	Provider WebRTC paths

Trigger a run

# Text mode (fastest for iteration)
cekura scenarios run-text \
  --agent-id 123 \
  --scenario-ids 1,2,3

# Voice via Cekura's hosted runner
cekura scenarios run-chirp \
  --agent-id 123 \
  --scenario-ids 1,2,3

# Self-hosted LiveKit
cekura scenarios run-livekit-v2 --from-file run.json

Each command returns a JSON envelope with a top-level result_id.

from cekura import Cekura

client = Cekura()

# Text mode
result = client.scenarios.run_text(
    agent_id=123,
    scenario_ids=[1, 2, 3],
)
result_id = result["result_id"]

Other modes follow the same shape:

client.scenarios.run_chirp(agent_id=123, scenario_ids=[1, 2, 3])
client.scenarios.run_livekit_v2(agent_id=123, scenario_ids=[1, 2, 3], ...)
client.scenarios.run_pipecat_v2(agent_id=123, scenario_ids=[1, 2, 3], ...)
client.scenarios.run_voice(agent_id=123, scenario_ids=[1, 2, 3], ...)
client.scenarios.run_sip(agent_id=123, scenario_ids=[1, 2, 3], ...)

Poll status

cekura runs list --result-id <result-id> --format json

Loop in a shell script:

while true; do
  STATUSES=$(cekura runs list --result-id "$RESULT_ID" --format json | jq -r '.[].status' | sort -u)
  [[ "$STATUSES" =~ pending|running ]] || break
  sleep 5
done

import time

while True:
    runs = client.runs.list(result_id=result_id)
    statuses = [r["status"] for r in runs.get("results", runs)]
    if all(s in ("passed", "failed", "errored", "cancelled") for s in statuses):
        break
    time.sleep(5)

print("done:", statuses)

Inspect a run

cekura runs get 5544
cekura runs list --agent-id 123

run = client.runs.get(run_id=5544)
print(run["status"])
print(run["transcript"])
for m in run["metric_results"]:
    print(m["metric_name"], m["value"])

Live operations on in-progress runs

# Get a live listen URL while a voice run is in progress
cekura runs listen-url 5544

# End an in-progress call
cekura runs end-call 5544

client.runs.get_listen_url(run_id=5544)
client.runs.end_call(run_id=5544)

Re-evaluate without re-running

If you change a metric prompt and want to score existing runs against the new definition:

cekura run rerun 987 --metric-ids 55,56

client.results.rerun(result_id=987, metric_ids=[55, 56])

Promote a run into a test set

Take a passing run and freeze it as a regression dataset:

cekura test-sets create-from-run --run-id 5544 --name "regression-2024-04"

client.test_sets.create_from_run(run_id=5544, name="regression-2024-04")

Evaluators

The scenarios that runs execute.

Metrics

Define how runs are scored.

Calls

Production calls — same scoring engine, different input.

API Reference

Full field reference for runs, results, and run-mode payloads.

CLI & SDK

Runs & Results

Pick a mode

Trigger a run

Poll status

Inspect a run

Live operations on in-progress runs

Re-evaluate without re-running

Promote a run into a test set

See also

Evaluators

Metrics

Calls

API Reference

CLI & SDK

Documentation Index

​Pick a mode

​Trigger a run

​Poll status

​Inspect a run

​Live operations on in-progress runs

​Re-evaluate without re-running

​Promote a run into a test set

​See also

Evaluators

Metrics

Calls

API Reference

Pick a mode

Trigger a run

Poll status

Inspect a run

Live operations on in-progress runs

Re-evaluate without re-running

Promote a run into a test set

See also