Create Metric - Cekura

cURL

curl --request POST \
  --url https://api.cekura.ai/test_framework/v1/metrics/ \
  --header 'Content-Type: application/json' \
  --header 'X-CEKURA-API-KEY: <api-key>' \
  --data '
{
  "name": "<string>",
  "description": "<string>",
  "audio_enabled": true,
  "prompt": "<string>",
  "assistant_id": "<string>",
  "type": "<string>",
  "eval_type": "<string>",
  "enum_values": {},
  "display_order": 123,
  "configuration": {},
  "agent": 123
}
'

{
  "id": 123,
  "project": 123,
  "agent": 123,
  "name": "<string>",
  "description": "<string>",
  "type": "basic",
  "eval_type": "binary_workflow_adherence",
  "enum_values": "<unknown>",
  "audio_enabled": true,
  "prompt": "<string>",
  "evaluation_trigger": "always",
  "evaluation_trigger_prompt": "<string>",
  "priority_assignment_prompt": "<string>",
  "configuration": "<unknown>",
  "overall_score": "<string>",
  "total_score": "<string>",
  "knowledge_base_files": "<string>",
  "observability_enabled": true,
  "simulation_enabled": true,
  "alert_enabled": true,
  "alert_type": "disabled",
  "significant_change_alert_status": "enabled",
  "significant_change_alert_direction": "",
  "function_name": "<string>",
  "custom_code": "<string>",
  "vocera_defined_metric_code": "<string>",
  "reviews": [
    {
      "id": 123,
      "test_set": {
        "id": 123,
        "agent": 123,
        "name": "<string>",
        "transcript": "<string>",
        "voice_recording_url": "<string>",
        "call_end_reason": "<string>",
        "duration": "<string>",
        "source_model": "CallLog",
        "source_id": "<string>",
        "created_at": "2023-11-07T05:31:56Z",
        "updated_at": "2023-11-07T05:31:56Z"
      },
      "metric": 123,
      "expected_value": "<unknown>",
      "actual_value": "<unknown>",
      "explanation": "<unknown>",
      "feedback": "<string>",
      "created_at": "2023-11-07T05:31:56Z",
      "updated_at": "2023-11-07T05:31:56Z"
    }
  ]
}

POST

test_framework

metrics

cURL

curl --request POST \
  --url https://api.cekura.ai/test_framework/v1/metrics/ \
  --header 'Content-Type: application/json' \
  --header 'X-CEKURA-API-KEY: <api-key>' \
  --data '
{
  "name": "<string>",
  "description": "<string>",
  "audio_enabled": true,
  "prompt": "<string>",
  "assistant_id": "<string>",
  "type": "<string>",
  "eval_type": "<string>",
  "enum_values": {},
  "display_order": 123,
  "configuration": {},
  "agent": 123
}
'

{
  "id": 123,
  "project": 123,
  "agent": 123,
  "name": "<string>",
  "description": "<string>",
  "type": "basic",
  "eval_type": "binary_workflow_adherence",
  "enum_values": "<unknown>",
  "audio_enabled": true,
  "prompt": "<string>",
  "evaluation_trigger": "always",
  "evaluation_trigger_prompt": "<string>",
  "priority_assignment_prompt": "<string>",
  "configuration": "<unknown>",
  "overall_score": "<string>",
  "total_score": "<string>",
  "knowledge_base_files": "<string>",
  "observability_enabled": true,
  "simulation_enabled": true,
  "alert_enabled": true,
  "alert_type": "disabled",
  "significant_change_alert_status": "enabled",
  "significant_change_alert_direction": "",
  "function_name": "<string>",
  "custom_code": "<string>",
  "vocera_defined_metric_code": "<string>",
  "reviews": [
    {
      "id": 123,
      "test_set": {
        "id": 123,
        "agent": 123,
        "name": "<string>",
        "transcript": "<string>",
        "voice_recording_url": "<string>",
        "call_end_reason": "<string>",
        "duration": "<string>",
        "source_model": "CallLog",
        "source_id": "<string>",
        "created_at": "2023-11-07T05:31:56Z",
        "updated_at": "2023-11-07T05:31:56Z"
      },
      "metric": 123,
      "expected_value": "<unknown>",
      "actual_value": "<unknown>",
      "explanation": "<unknown>",
      "feedback": "<string>",
      "created_at": "2023-11-07T05:31:56Z",
      "updated_at": "2023-11-07T05:31:56Z"
    }
  ]
}

Authorizations

X-CEKURA-API-KEY

string

header

required

API Key Authentication. It should be included in the header of each request.

Body

name

string

required

Name of the metric

description

string

required

Description of what this metric evaluates

audio_enabled

boolean

required

Whether this metric evaluates audio content

prompt

string

required

The evaluation prompt used for this metric

assistant_id

string

required

External identifier for the assistant

type

string

required

Type of metric (e.g., basic, custom_prompt, custom_code)

eval_type

string

required

Type of evaluation (e.g., boolean, enum, score)

enum_values

object

required

Possible values for enum-type metrics

Show child attributes

enum_values.{key}

any

display_order

integer

required

Order in which to display this metric in the UI

configuration

object

required

Custom configuration parameters for specific metrics. For pronounciation metric, you can set words as 2-tuple (word, phonemes) list example:

{
    "words": [["hello", "hɛl.loʊ"], ["world", "wɝɚɚɚld"]]
}

Show child attributes

configuration.{key}

any

agent

integer

The AI agent this metric is associated with

Response

integer

project

integer | null

agent

integer | null

name

string

Name of the metric. Example: "Customer Satisfaction" or "Appointment Booking"

Maximum string length: 255

description

string

Description of what the metric measures. Example: "Measures how satisfied customers are with the service provided"

type

enum<string>

basic - Basic
custom_prompt - Custom Prompt
custom_code - Custom Code

Available options:

basic,

custom_prompt,

custom_code

eval_type

enum<string>

binary_workflow_adherence - Binary Workflow Adherence
binary_qualitative - Binary Qualitative
continuous_qualitative - Continuous Qualitative
numeric - Numeric
enum - Enum

Available options:

binary_workflow_adherence,

binary_qualitative,

continuous_qualitative,

numeric,

enum

enum_values

any

List of possible enum values for enum type metrics. Example: ["satisfied", "unsatisfied"]

audio_enabled

boolean

Whether this metric requires audio analysis. Example: true or false

prompt

string

Evaluation prompt for the metric. Example: "Evaluate customer satisfaction based on conversation"

evaluation_trigger

enum<string>

always - Always
automatic - Automatic
custom - Custom

Available options:

always,

automatic,

custom

evaluation_trigger_prompt

string

Evaluation trigger prompt for the metric. Example: "Evaluate metric only if call ended reason is main-agent-ended-call"

priority_assignment_prompt

string

Priority assignment prompt for the metric.

configuration

any

Custom configuration parameters for specific metrics if metric supports it. Example:

For Infrastructure issues

{
    "infra_issues_timeout": 10
}

overall_score

string

The overall score for this metric across all test sets

total_score

string

The total score for this metric

knowledge_base_files

string

Knowledge base files associated with this metric

observability_enabled

boolean

Enable this metric for observability. Example: true or false

simulation_enabled

boolean

Enable this metric for simulations. Example: true or false

alert_enabled

boolean

Enable alerts for this metric when it fails (value < 5). Only applicable to binary metrics (binary_workflow_adherence and binary_qualitative). For other metric types, use significant_change_alert_status instead. Example: true or false

alert_type

enum<string>

default:disabled

disabled - Alerts Disabled
normal - Normal Alerts
significant_change - Significant Change Alerts

Available options:

disabled,

normal,

significant_change

significant_change_alert_status

enum<string>

Alert status: enabled or disabled.

enabled - Enabled
disabled - Disabled

Available options:

enabled,

disabled

significant_change_alert_direction

enum<string>

Alert direction: increase only, decrease only, or both (empty = both). Example: "increase", "decrease", or "both"

`` - Both (Increase and Decrease)
increase - Increase Only
decrease - Decrease Only

Available options:

increase,

decrease

function_name

string | null

Predefined function name Example: "get_latency" or "check_critical_deviations"

Maximum string length: 255

custom_code

string

Python custom code for the metric. Example:

_resul = True
_explanation = None
if  "call_end_reason" in data and data["call_end_reason"] == "customer_satisfaction":
    _result = True
    _explanation = "Customer expressed satisfaction with service"

vocera_defined_metric_code

string

Vocera defined metric code for the metric. Example: "7fd534f5"

Maximum string length: 255

reviews

object[]

Reviews associated with the metric

Show child attributes

reviews.id

integer

reviews.test_set

object

Show child attributes

reviews.test_set.id

integer

reviews.test_set.agent

integer

reviews.test_set.name

string

Name of the test set Example: "Test Set 1"

Maximum string length: 255

reviews.test_set.transcript

string

Full text transcript of the call. Example:

[00:01] Testing Agent: Hello. 
[00:02] Main Agent: Hello, how can I help you today?
[00:03] Testing Agent: Well, I mean, sure. What time exactly are we talking about here
[00:04] Main Agent: 6 PM.
[00:05] Testing Agent: Great. I'll book that for you. Just a sec.
[00:06] Main Agent: Okay.

reviews.test_set.voice_recording_url

string

reviews.test_set.call_end_reason

string

Reason why the call ended. Example:

"customer-ended-call"
"agent-ended-call"

Maximum string length: 255

reviews.test_set.duration

string

reviews.test_set.source_model

enum<string>

CallLog - Call Log
Run - Run

Available options:

CallLog,

Run

reviews.test_set.source_id

string

reviews.test_set.created_at

string<date-time>

When this record was created. Example: 2024-03-15T10:30:45Z

reviews.test_set.updated_at

string<date-time>

When this record was last updated. Example: 2024-03-15T10:35:11Z

reviews.metric

integer

reviews.expected_value

any | null

Metric expected value Example:

5 or 0
one, two or three [ enum type ]

reviews.actual_value

any | null

Metric actual value Example:

5 or 0
one, two or three [ enum type ]

reviews.explanation

any

Metric explanation Example:

["Agent failed to follow the opening script", "Missed key product information"]

reviews.feedback

string

Metric feedback Example:

"Great Call"

reviews.created_at

string<date-time>

Timestamp when the metric review was created Example: "2021-01-01T00:00:00Z"

reviews.updated_at

string<date-time>

Timestamp when the metric review was last updated Example: "2021-01-01T00:00:00Z"

List Metrics Update Metric

⌘I

API Reference

Authorizations

Body

Response