List Runs with IDs

cURL

curl --request GET \
  --url https://api.cekura.ai/test_framework/v1/runs/bulk/ \
  --header 'X-CEKURA-API-KEY: <api-key>'

[
  {
    "id": 123,
    "provider_call_id": "<string>",
    "scenario": 123,
    "scenario_name": "Scenario Deleted",
    "personality": 123,
    "personality_name": "<string>",
    "agent": 123,
    "expected_outcome_prompt": "<string>",
    "expected_outcome": {
      "score": 123,
      "explanation": [
        "<string>"
      ],
      "outcome_alignments": [
        {
          "outcome": "<string>",
          "prompt_part": "<string>",
          "aligned": true
        }
      ]
    },
    "evaluation": {
      "metrics": [
        {
          "id": 123,
          "name": "<string>",
          "type": "<string>",
          "score": 123,
          "enum": "<string>",
          "value": "<string>",
          "explanation": [
            "<string>"
          ],
          "extra": {}
        }
      ]
    },
    "transcript": "<string>",
    "transcript_object": [
      {
        "role": "<string>",
        "content": "<string>",
        "timestamp": 123
      }
    ],
    "duration": "<string>",
    "voice_recording_url": "<string>",
    "email_id": "[email protected]",
    "status": "running",
    "evaluation_status": "success",
    "metadata": "<unknown>",
    "success": true,
    "is_reviewed": true,
    "feedback": "<string>",
    "timestamp": "2023-11-07T05:31:56Z",
    "executed_at": "2023-11-07T05:31:56Z",
    "outbound_number": "<string>",
    "critical_categories": [
      {
        "category_id": 123,
        "category": "<string>",
        "deviation": 123,
        "wrongly_marked": true,
        "feedback": "<string>",
        "priority": "critical"
      }
    ],
    "error_message": "<string>",
    "scenario_instructions": "<string>",
    "run_as_text": true,
    "test_profile_data": {
      "id": 123,
      "name": "<string>",
      "information": {}
    },
    "provider_call_details": "<unknown>"
  }
]

GET

test_framework

runs

bulk

cURL

curl --request GET \
  --url https://api.cekura.ai/test_framework/v1/runs/bulk/ \
  --header 'X-CEKURA-API-KEY: <api-key>'

[
  {
    "id": 123,
    "provider_call_id": "<string>",
    "scenario": 123,
    "scenario_name": "Scenario Deleted",
    "personality": 123,
    "personality_name": "<string>",
    "agent": 123,
    "expected_outcome_prompt": "<string>",
    "expected_outcome": {
      "score": 123,
      "explanation": [
        "<string>"
      ],
      "outcome_alignments": [
        {
          "outcome": "<string>",
          "prompt_part": "<string>",
          "aligned": true
        }
      ]
    },
    "evaluation": {
      "metrics": [
        {
          "id": 123,
          "name": "<string>",
          "type": "<string>",
          "score": 123,
          "enum": "<string>",
          "value": "<string>",
          "explanation": [
            "<string>"
          ],
          "extra": {}
        }
      ]
    },
    "transcript": "<string>",
    "transcript_object": [
      {
        "role": "<string>",
        "content": "<string>",
        "timestamp": 123
      }
    ],
    "duration": "<string>",
    "voice_recording_url": "<string>",
    "email_id": "[email protected]",
    "status": "running",
    "evaluation_status": "success",
    "metadata": "<unknown>",
    "success": true,
    "is_reviewed": true,
    "feedback": "<string>",
    "timestamp": "2023-11-07T05:31:56Z",
    "executed_at": "2023-11-07T05:31:56Z",
    "outbound_number": "<string>",
    "critical_categories": [
      {
        "category_id": 123,
        "category": "<string>",
        "deviation": 123,
        "wrongly_marked": true,
        "feedback": "<string>",
        "priority": "critical"
      }
    ],
    "error_message": "<string>",
    "scenario_instructions": "<string>",
    "run_as_text": true,
    "test_profile_data": {
      "id": 123,
      "name": "<string>",
      "information": {}
    },
    "provider_call_details": "<unknown>"
  }
]

Authorizations

X-CEKURA-API-KEY

string

header

required

API Key Authentication. It should be included in the header of each request.

Query Parameters

run_ids

string

required

Comma-separated list of run IDs. Example: 1,2,3

Response

200 - application/json

integer

provider_call_id

string

Call ID from the Assistant provider Example: call_abc123

scenario

integer | null

scenario_name

string

default:Scenario Deleted

personality

integer

Personality this run belongs to

personality_name

string

Name of the personality this run belongs to

agent

integer

ID of the agent associated with this run

expected_outcome_prompt

string

expected_outcome

object

Show child attributes

expected_outcome.score

number<double>

Score of the expected outcome

expected_outcome.explanation

string[]

Explanation of the expected outcome

expected_outcome.outcome_alignments

object[]

Show child attributes

expected_outcome.outcome_alignments.outcome

string

Outcome of the expected outcome

expected_outcome.outcome_alignments.prompt_part

string

Prompt part of the expected outcome

expected_outcome.outcome_alignments.aligned

boolean

Whether the outcome aligns with the prompt part

evaluation

object

Run Evaluation

Show child attributes

evaluation.metrics

object[]

Show child attributes

evaluation.metrics.id

integer

Metric ID

evaluation.metrics.name

string

Metric name

evaluation.metrics.type

string

Metric type

evaluation.metrics.score

number<double>

Evaluated metric score, Present if metrictype is not enum

evaluation.metrics.enum

string

Metric enum, Present if type is enum

evaluation.metrics.value

string

Evaluated metric value, Present if metric type is enum

evaluation.metrics.explanation

string[]

Explanation of the metric

evaluation.metrics.extra

object

Evaluated metric extra data

Show child attributes

evaluation.metrics.extra.{key}

any

transcript

string

Full text transcript of the call. Example:

[00:01] Testing Agent: Hello. 
[00:02] Main Agent: Hello, how can I help you today?
[00:03] Testing Agent: Well, I mean, sure. What time exactly are we talking about here
[00:04] Main Agent: 6 PM.
[00:05] Testing Agent: Great. I'll book that for you. Just a sec.
[00:06] Main Agent: Okay.

transcript_object

object[]

Transcript object

Show child attributes

transcript_object.role

string

Role of the message

transcript_object.content

string

Content of the message

transcript_object.timestamp

number<double>

Timestamp of the message

duration

string

Duration of the run in MM:SS format

voice_recording_url

string

URL to access the voice recording of this run

email_id

string<email> | null

status

enum<string>

running - Running
completed - Completed
failed - Failed
pending - Pending
in_progress - In Progress
evaluating - Evaluating
in_queue - In Queue
timeout - Timeout
cancelled - Cancelled

Available options:

running,

completed,

failed,

pending,

in_progress,

evaluating,

in_queue,

timeout,

cancelled

evaluation_status

enum<string>

success - Success
failure - Failure
reviewed_success - Reviewed Success
reviewed_failure - Reviewed Failure

Available options:

success,

failure,

reviewed_success,

reviewed_failure

metadata

any

success

boolean

is_reviewed

boolean

Whether the call has been reviewed. Example: true or false

feedback

string

timestamp

string<date-time>

executed_at

string<date-time> | null

outbound_number

string

Phone number used to make the outbound call for this run

critical_categories

object[]

List of critical evaluation categories and their results for this run

Show child attributes

critical_categories.category_id

integer

Category ID

critical_categories.category

string

Category name

critical_categories.deviation

number<double>

Deviation

critical_categories.wrongly_marked

boolean

Whether the category was wrongly marked

critical_categories.feedback

string

Feedback

critical_categories.priority

enum<string>

Priority

critical - Critical
non-critical - Non Critical
not_a_bug - Not A Bug

Available options:

critical,

non-critical,

not_a_bug

error_message

string

scenario_instructions

string

run_as_text

boolean

test_profile_data

object

Test Profile

Show child attributes

test_profile_data.id

integer

Test Profile ID

test_profile_data.name

string

Test Profile name

test_profile_data.information

object

Information of the test profile Example:

{
    "user_name": "John Doe",
    "user_email": "[email protected]",
}

Show child attributes

test_profile_data.information.{key}

any

provider_call_details

any

Details of the call from the provider

Generate shareable link for result Runs Improve Prompt

⌘I

API Reference

Authorizations

Query Parameters

Response