Chat-Based Testing

Overview

Test your voice agents using text instead of phone calls. 10x faster and 90% cheaper - ideal for workflow validation, regression testing, and CI/CD pipelines. Use voice testing for ASR/TTS validation and final production checks.

Supported Providers

Configure Chat Provider

From sidebar, go to Agent → Configuration → Provider, select ElevenLabs
Under Integration Settings, enter your External Assistant ID and ElevenLabs API Key
Click Save Changes

Run Tests

Go to Evaluator page, select scenarios and click on Run button.
In the Configure Run dialog, select ElevenLabs under Chat connections and click Run

Greyed out connection options have not been configured in Agent Settings yet — set them up there to enable them.

View Results

Results appear in the dashboard with conversation transcript, metric evaluations, and function calls.

For voice Personas, automatic configuration, outbound calls, and mock tools, see the Bland AI integration guide.

Create Pathway in Bland

Log in to Bland dashboard
Navigate to Conversational Pathways
Click “Create Pathway” or duplicate a template
Build your conversation flow using nodes and edges
Test and publish your pathway

Get Pathway ID

In Bland dashboard, open your pathway
Copy the Pathway ID from the URL or pathway settings
Format: UUID like 9d404c1b-6a23-4426-953a-a52c392ff8f1

Configure in Cekura

Go to Agent Settings → Configuration → Provider, select Bland AI
Under Integration Settings, enter the External Assistant ID (your Pathway ID) and Bland API Key
Click Save Changes

Run Tests

Go to Evaluator page and select scenarios
In the Configure Run dialog, select Bland under Chat connections and click Run

Configure Run dialog with Bland chat connection selected

Greyed out connection options have not been configured in Agent Settings yet — set them up there to enable them.

Set Up WebSocket Server

Use the example implementation or build your own:

git clone https://github.com/vocera-ai/llm-websocket-server-example.git
cd llm-websocket-server-example
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python main.py

Update your OpenAI API key in main.py before starting the server.

Authentication:When Cekura connects to your WebSocket server, it includes the following headers in the handshake request:

Header	Description
`X-VOCERA-SECRET`	The Websocket Secret you configured in your agent’s Custom Chatbot Settings. Cekura sends this exact value on every connection so your server can verify the request came from Cekura.
`X-VOCERA-SCENARIO-ID`	The ID of the scenario being tested.
`X-VOCERA-RESULT-ID`	The result ID for this test run.
`X-VOCERA-RUN-ID`	The run ID for this test execution.

Any custom headers configured in your agent’s WebSocket header settings are also included.To authenticate incoming connections, validate the X-VOCERA-SECRET header against the secret you configured in your agent’s Custom Chatbot Settings:

# The same value you saved as "Websocket Secret" in Agent Settings → Chatbot Integrations
WEBSOCKET_SECRET = "your-configured-websocket-secret"

async def handle_websocket(websocket, path):
    secret = websocket.request_headers.get("X-VOCERA-SECRET")
    if secret != WEBSOCKET_SECRET:
        await websocket.close(1008, "Unauthorized")
        return
    # Handle messages...

WebSocket Message Format:Messages your server sends to Cekura (your agent’s messages):Regular message:

{
  "content": "Hello! How can I help you today?"
}

Message with metadata:

{
  "content": "Hello! How can I help you today?",
  "metadata": {
    "customer_id": "cust_123",
    "session_type": "support"
  }
}

Metadata-only message (no conversation response triggered):

{
  "metadata": {
    "internal_call_id": "abc-123",
    "region": "us-west-2"
  }
}

Function call:

{
  "role": "Function Call",
  "data": {
    "id": "call_abc123",
    "name": "get_weather",
    "arguments": "{\"location\": \"New York\"}"
  }
}

Function call result:

{
  "role": "Function Call Result",
  "data": {
    "id": "call_abc123",
    "result": "{\"temperature\": 72, \"unit\": \"fahrenheit\"}"
  }
}

End call (server-initiated session end):

{
  "content": "Goodbye, have a great day!",
  "type": "end_call"
}

To end the conversation from your server, send a message with "type": "end_call". Cekura will finalize the session on receipt. This is the same message format Cekura uses when it initiates the end — either side can send it.

Messages Cekura sends to your server (testing agent messages):Regular message:

{
  "content": "I'd like to book an appointment for tomorrow"
}

End call message:

{
  "content": "Thank you, goodbye!",
  "type": "end_call"
}

When your server receives a message with "type": "end_call", the conversation is ending. Your server can handle cleanup and connection closure gracefully.

Conversation Finalization & Timeouts:Either side can end the conversation. Cekura sends {"type": "end_call"} to your server and waits 1 second before closing the connection — your server does not need to close the connection itself. To end the conversation from your side, simply close the WebSocket connection; Cekura treats an unexpected close as the main agent ending the call.Cekura closes the connection after 5 minutes without a content message. It also sends a WebSocket ping every 30 seconds and considers the connection dropped if no pong is received within 10 seconds. Ping/pong are handled automatically at the WebSocket protocol layer by most libraries, so no explicit handling is required in your application code.

Expose Server

For local development, use ngrok:

ngrok http 127.0.0.1:8765

Convert the https:// URL to wss:// for WebSocket connection.

Configure in Cekura

Go to Agent Settings → Configuration → Connection → Chat Connection → WebSocket
Enter your WebSocket URL (starting with wss://)
Enter WebSocket Secret (Optional) — any string you choose. Cekura will send this value on every connection in the X-VOCERA-SECRET header so your server can authenticate the request. Use the Update button to change it later.
Click Save Changes

The Websocket Secret is a value you choose. Keep the same value in your WebSocket server so it can validate the X-VOCERA-SECRET header. See the Authentication subsection under Set Up WebSocket Server above for a code example.

Run Tests

Go to Evaluator page and select scenarios
In the Configure Run dialog, select Websocket under Chat connections and click Run

Configure Run dialog with Websocket chat connection selected

Greyed out connection options have not been configured in Agent Settings yet — set them up there to enable them.

Test Profile Headers

When test profile is attached on your Evaluator and run it as WebSocket Chat, any fields in the test profile starting with X- are automatically sent as HTTP headers to your WebSocket server during connection establishment. This allows your server to receive user context before the conversation begins.Example Test Profile:

{
  "name": "Sarah Johnson",
  "email": "sarah.j@email.com",
  "X-Customer-ID": "CUST-98765",
  "X-Account-Tier": "premium"
}

Headers Your Server Receives:

X-Customer-ID: CUST-98765
X-Account-Tier: premium
X-VOCERA-SECRET: <your-configured-websocket-secret>
X-VOCERA-RUN-ID: <run-id>
X-VOCERA-SCENARIO-ID: <scenario-id>
X-VOCERA-RESULT-ID: <result-id>

Python WebSocket Server Example:

async def handle_connection(websocket, path):
    headers = websocket.request_headers

    customer_id = headers.get('X-Customer-ID')
    account_tier = headers.get('X-Account-Tier')

    # Pre-load customer data before conversation starts
    customer_data = await fetch_customer(customer_id)

    # Customize behavior based on tier
    if account_tier == 'premium':
        enable_priority_support()

    # Now handle the conversation...

Only fields starting with X- are sent as headers. Other fields remain available for use in evaluator instructions via template variables like {{test_profile.name}}.

Use Cases:

Pre-load customer/user data before conversation starts
Route to specific handlers based on account tier or customer type
Initialize session context with user preferences

Metadata

Your WebSocket server can attach metadata to a run by including a metadata field in any message sent to Cekura. Metadata is useful for passing contextual information like internal IDs, environment details, or customer attributes that you want associated with the test run.Sending metadata with a message:

{
  "content": "Hello! How can I help you today?",
  "metadata": {
    "customer_id": "cust_123",
    "session_type": "support"
  }
}

Sending metadata without a message:You can send a metadata-only message (no role or content) at any point during the conversation. This attaches the metadata to the run without triggering a response from the testing agent:

{
  "metadata": {
    "internal_call_id": "abc-123",
    "region": "us-west-2"
  }
}

Behavior:

Metadata must be a JSON object. Non-object values are ignored.
Metadata is merged across messages using last-write-wins — if you send the same key in multiple messages, the latest value is kept.

Python example:

import json

async def handle_connection(websocket, path):
    # Send metadata at the start of the conversation
    await websocket.send(json.dumps({
        "metadata": {
            "internal_session_id": "sess_abc",
            "environment": "staging"
        }
    }))

    async for message in websocket:
        data = json.loads(message)
        response = generate_response(data["content"])

        # Send response with additional metadata
        await websocket.send(json.dumps({
            "content": response,
            "metadata": {
                "model_used": "gpt-4o",
                "response_latency_ms": 342
            }
        }))

Start the run

curl --request POST \
  --url https://api.cekura.ai/test_framework/v1/scenarios/run_scenarios_api_chat/ \
  --header 'X-CEKURA-API-KEY: <api-key>' \
  --header 'Content-Type: application/json' \
  --data '{ "agent_id": 123, "scenarios": [456], "frequency": 1, "timeout": 1800 }'

Each runs[].id in the response is a conversation to drive. timeout is the overall run lifetime in seconds (defaults to your project setting, max 3600).

Wait until the run is ready

Poll the run until its status is in_progress (it opens once capacity is available):

curl --request GET \
  --url https://api.cekura.ai/test_framework/v1/runs/<run_id>/ \
  --header 'X-CEKURA-API-KEY: <api-key>'

Then fetch the opening as turn 0 — GET /runs/<run_id>/chat/0/ returns the scenario’s opening line in message (the simulated user speaks first), or null when you speak first.

Submit a turn

Send your agent’s latest message. The reply is generated in the background, so this returns a turn_id to poll.

curl --request POST \
  --url https://api.cekura.ai/test_framework/v1/runs/<run_id>/chat/ \
  --header 'X-CEKURA-API-KEY: <api-key>' \
  --header 'Content-Type: application/json' \
  --data '{ "content": "Hello! How can I help you today?" }'

{ "run_id": 987, "result_id": 654, "turn_id": 1, "status": "generating" }

Submitting a new turn while the previous one is still generating interrupts it — the older turn becomes dropped but its message is kept, and the new reply answers all messages received since the last reply.

Poll for the reply

Poll the turn until status is ready; the reply is in message. Feed it to your agent and submit the next turn. Repeat until ended is true.

curl --request GET \
  --url https://api.cekura.ai/test_framework/v1/runs/<run_id>/chat/<turn_id>/ \
  --header 'X-CEKURA-API-KEY: <api-key>'

{ "run_id": 987, "result_id": 654, "turn_id": 1, "status": "ready",
  "message": "Hi! I'd like to check the status of my order.", "ended": false, "ended_reason": null }

Statuses: generating (keep polling), ready (reply in message), dropped (interrupted by a newer turn — its message is kept and the newer reply answers it), error (submit again). Turns stay pollable after the run finishes.

Finish (optional)

End explicitly to score it right away; otherwise it finalizes when the simulated user ends the chat or timeout elapses.

curl --request POST \
  --url https://api.cekura.ai/test_framework/v1/runs/<run_id>/end-chat/ \
  --header 'X-CEKURA-API-KEY: <api-key>' \
  --header 'Content-Type: application/json' \
  --data '{ "reason": "client-finished" }'

Results appear in the dashboard with the full transcript and metrics, like any other run.

Run chat tests programmatically via API.For complete API documentation including authentication, request parameters, response format, and code examples in multiple languages, see:Run Evaluator Text API Reference

Quick Example

curl --request POST \
  --url https://api.cekura.ai/test_framework/v1/scenarios/run_scenarios_text/ \
  --header 'X-CEKURA-API-KEY: <api-key>' \
  --header 'Content-Type: application/json' \
  --data '{
    "agent_id": 123,
    "scenarios": [456, 789],
    "frequency": 1,
    "name": "Chat Test Run"
  }'

Troubleshooting

Error: "Chat provider not configured"

Solution:

Go to Agent Settings → Chatbot Integrations
Select a provider (ElevenLabs, Retell, Vapi, or Custom)
Enter required configuration (Assistant ID or WebSocket URL)
Save and retry

Tests timing out

Common causes:

Incorrect Assistant/Agent ID
Custom WebSocket server not accessible
Agent disabled or unpublished

Debug steps:

Verify your ID is correct
For Custom: check server is running and URL is accessible
Test provider’s API directly
Check Cekura dashboard for error details

Cannot find "Run as Text" button

Checklist:

✅ Chat provider configured in Agent Settings
✅ On Evaluator page (not Calls/Runs)
✅ At least one scenario selected

Button appears next to “Run Tests” in Evaluator header. Refresh page if not visible.

Tips

Use chat tests for rapid iteration and CI/CD. Run voice tests on critical scenarios before production deployment.

Tag scenarios as “critical” or “smoke-test” to run targeted test suites in your pipeline.

Next Steps

Need help? Contact support@cekura.ai

Get Started

Key Concepts

Guides

Integrations

Advanced

Chat-Based Testing

Overview

Supported Providers

Test Profile Headers

Metadata

Quick Example

Troubleshooting

Tips

Next Steps

​Overview

​Supported Providers

​Test Profile Headers

​Metadata

​Quick Example

​Troubleshooting

​Tips

​Next Steps

Overview

Supported Providers

Test Profile Headers

Metadata

Quick Example

Troubleshooting

Tips

Next Steps