Improve Agent is BETA. It supports agents on VAPI, Retell, ElevenLabs and Bland. The loop reads and edits the agent’s configuration through your provider credentials, so the agent must already be connected to Cekura.
Where to start it
Four places in the dashboard launch the same loop. What differs is where the failures come from.1
Insights — a failure category
Each failure category on the Insights page has its own action. Click Improve agent on the category you care about. If that category’s calls span more than one agent, you’ll be asked which agent to improve first. The loop writes fresh evaluator scenarios from the production calls behind that category, then works from those.
2
A run or call detail page
Open a run and click Improve Agent on the results card. It uses every metric on that run — the failing ones are what it tries to fix, and the passing ones become guardrails it must not break. You don’t pick metrics.
3
A result's failure breakdown
On a finished result, launch straight from the failure breakdown. The scores you’re already looking at are reused as the starting measurement instead of paying to re-run the same conversations.
4
Evaluators list — bulk action
Select one or more evaluators and use the Improve Agent bulk action. Those evaluators become the validation set.
Before you start
1
The agent is connected to Cekura
The loop reads your agent’s live configuration to know what it can change. See Cekura Agent.
2
There is something to measure against
Your project needs either a rubric rule or at least one metric enabled for simulations. Without one, there is no way to score a before and after, and the run is refused up front rather than burning credits.
3
There is a real failure
The loop’s input is failures. If everything already passes, it stops and tells you so instead of changing things for their own sake.
What the loop does
1
Makes a private copy
Your agent is copied at your provider. Every change and every test conversation from here on happens against that copy. Your live agent is untouched for the entire run.
2
Writes scenarios (production-call runs only)
When the failures come from real calls, the loop reads those transcripts and writes evaluator scenarios that reproduce them. When you started from a run or from evaluators, the scenarios already exist and this step is skipped.
3
Reproduces the failure (production-call runs only)
Before changing anything, the loop proves it can make the problem happen on demand. See Reproducing the failure.
4
Measures a baseline
Runs the scenarios against the copy and records the starting score for every metric. This is the “before” side of everything you’re shown later.
5
Proposes a change
Reads the failing transcripts and the current instructions, and proposes the smallest set of edits that would fix what actually went wrong. If the answer depends on a business rule it can’t infer — which of two policies is correct, what the agent may promise — it stops and asks you rather than guessing.
6
Applies and re-tests
Lands the edits on the copy and re-runs the scenarios that were failing. If they all pass, it re-runs the rest to check nothing regressed.
7
Repeats, then reports
Loops until everything passes or it stops making progress, then shows you the before/after per metric, the exact changes, and a Promote button.
What it is allowed to change
The loop edits the agent’s instructions and closely related configuration on the copy. Some things are permanently off-limits and are rejected before they reach your provider:- credentials and API keys
- webhook and server URLs
- phone numbers
- transfer and handoff destinations
- voice and account wiring
Reproducing the failure
A failure that happened in production doesn’t always happen in a test. When you start from production calls, the loop treats reproducing the problem as a precondition — not as an optional first attempt. It writes scenarios from the failing calls — as few as will cover them, since every cited metric is scored on every conversation — and runs each of them once. The failure counts as reproduced as soon as the metric that flagged those calls in production fails on one of those conversations: that is all the loop needs before it can start measuring whether a change fixes it. The check is arithmetic, not a judgement call: conversations that were never scored — a timed-out call, a scoring error — count as neither evidence for nor against. Saying the opposite takes more evidence than that, deliberately. “We couldn’t reproduce it” either ends the run or sends it to the last resort below, so the loop won’t conclude it until enough conversations have actually been scored to stand behind it. Those same conversations are your baseline. Rather than buy a second, identical batch to measure where the agent starts, the loop scores the ones it just ran — so proving the failure and establishing the “before” number cost one round of testing between them, not two. If the failure doesn’t show up, the loop doesn’t give up on the first try. It:- Tries again with sharper scenarios. It’s told what it already tried and that it didn’t reproduce, then re-reads the transcripts to find what the real calls had that its tests didn’t.
- Tries to force the failure. If repeated attempts still can’t provoke it, the loop may deliberately weaken a safeguard on the private copy to make the latent problem surface every time. This is a testing device, never a suggestion.
A change made only to force the failure is never offered to you and can never be promoted. Before the run finishes, the loop puts the safeguard back and re-runs your whole scenario set on the result — so the version you’re offered is the one that was actually tested. If the fix only worked while the safeguard was removed, you’re told exactly that, and nothing is offered.
Accepting the changes
Nothing reaches your live agent until you click Promote.- Promote applies the offered changes to your live agent. It re-checks first that the copy still matches what was measured and that your live agent hasn’t drifted since the run started; if either has changed, it refuses rather than overwriting work you did in the meantime.
- Discard copy deletes the private copy at your provider.
- Copies are cleaned up automatically after about a week, and immediately when a run ends with nothing to offer.
While it’s running
- Answer its questions. When the loop pauses on a business-rule question, reply in the thread and it picks up where it left off.
- Steer it. You can send guidance mid-run — context about your product, a constraint, a direction to try — and it’s folded into the next round.
- Stop it. Cancel at any point. The copy is cleaned up.
How it ends
Defaults: up to 15 rounds, stopping early after 5 rounds without improvement, and at most 5 clarifying questions per run.
Troubleshooting
The action doesn’t appear on an insight. That insight’s calls belong to an agent on a provider the loop can’t edit. Use Create scenario instead — it still turns those calls into evaluators you can run and fix by hand. It says my agent already passes everything. The scenarios it ran didn’t fail. If the failure is real but intermittent, that’s the reproduction problem described above; starting from the Insights category rather than a single call gives it more transcripts to work from. Most failures cluster on one metric whose explanations look subjective. The metric is probably miscalibrated, and improving the agent against a noisy metric chases noise. Hand off to the Metric Lab improvement flow first. Promote was refused. Either your live agent changed since the run started, or the copy no longer matches what was measured. Both mean the offered before/after no longer describes reality. Re-run rather than forcing it. The run is refused before it starts. Your project has no rubric rule and no metric enabled for simulations, so there’s nothing to score against. See Metrics. It stopped after writing the scenarios, saying it can’t score the checks that flagged those calls. The metrics named on those insights no longer resolve to a metric in the project — deleted, renamed, or two metrics sharing one name — so nothing would have scored the failure and every test conversation would have been paid for with no verdict to show. The scenarios it wrote stay in your evaluators list, so fixing the metric and starting again reuses them.Self-hosted and custom agents
Agents that Cekura can’t edit through a provider API — self-hosted, websocket and custom integrations — aren’t supported by the loop above, because it depends on being able to copy the agent and change it. For those, ask the AI Assistant to improve your agent from a set of evaluators: it runs the same diagnose-and-retest reasoning and hands you the instruction changes to deploy yourself, then re-tests once you’ve deployed.Related
- Cekura Agent — connect an agent before optimizing it.
- Insights — where production failures get grouped into categories.
- Metric Lab — improve evaluator quality when failures cluster on a noisy metric.
- Auto-optimise Metrics — the equivalent recurring workflow for metric prompts.