Is Your AI Agent Still Working?

A few months after go-live, the screening agent in a know-your-customer (KYC) onboarding process has handled several hundred cases. During this time, the LLM provider shipped a new model version, and the team edited the system prompt twice. Then the monthly review comes around, and the center of excellence (COE) lead asks three questions about the agent: is it drifting, what is it spending, and did the last change make it better or worse?

None of those has an answer inside a single process instance. Operate explains one case: what an agent is doing right now and why it did what it did. The COE lead is asking about the agent's behavior across all of its cases, and a question about all of them needs a trend to answer it.

Losing sight of agents after go-live

An agent rarely fails loudly in production. After a prompt edit it writes longer answers, after a model update it checks one more source per case, and every case still completes. Nothing in a single instance looks wrong, so the change goes unnoticed until someone reviews the total cost of ownership and the quality metrics, often months later.

That's where the risk sits. Token spend creeps up and gets discovered on the invoice. Quality slips and gets discovered by a customer or an auditor. A model swap ships with no evidence that it helped, and nobody can say whether to keep it or roll it back. And when the COE can't answer basic questions about the agents already in production, it stops trusting them and holds back the next ones, because every new agent adds a risk nobody can measure. According to Camunda's 2026 State of Agentic Orchestration and Automation Report, 71% of organizations use AI agents, but only 11% of use cases reached production last year.

Most teams fill the gap by hand: exporting traces, copying metrics into spreadsheets, building a dashboard per agent. That works for one agent and breaks down once agents run inside dozens of processes, built on more than one framework.

Measuring agents as part of the process

Camunda 8.10 adds the Agentic control plane to Optimize: one dashboard that shows how the agents in your processes behave over time, across the whole cluster, or for a single process and version.

What makes it different is where the numbers come from. Tools that observe only the agent can tell you what the agent did. Camunda orchestrates agents both from the outside and the inside of a business process. For Camunda AI agents, it enforces steps between the agent’s decision and its action,=,

The control plane captures what happens with both Camunda AI agents and external agents, measuring each one in the context of the business case it served, next to the people and systems who handled the rest of it.

In the KYC process from earlier in this series, a built-in approval step can hold the screening agent’s decision for a person to sign off, and the control plane shows that step next to the agent’s own behavior.

Answering the review with the control plane

A monthly review usually starts wide and narrows down: how is agentic work doing overall, which process needs attention, and what exactly changed inside it. The control plane follows the same path, so the COE lead can walk into the review with answers instead of a list of tickets for the engineering team.

It opens on the whole cluster. The top row shows how many agentic process instances completed, how long they took, and how often they hit an incident, each with its change since last month. That's the health check: whether agentic work is growing, and whether it's getting slower or less reliable as it grows. Below it, the dashboard groups the detail into token usage, tool calls and reliability, and duration. Select a process and a version, and every section narrows to it.

That structure is what lets one dashboard answer each of the review questions.

Is it drifting? Drift shows up in the shape of the numbers before it shows up as incidents. When a few cases start costing far more than the typical one, the average tokens per execution pull away from the median, and the band between the cheapest and the most expensive cases widens. When the agent starts looping or a tool stops returning useful answers, tool calls climb while case volume stays flat. When the slowest cases get slower, the gap between median and 95th percentile duration grows. You catch the drift at the review, while it's still a trend and well before it reaches an incident report or an invoice.

What is it spending? Across the cluster, the control plane ranks processes by the tokens they consume, so you know which one is worth looking at first. Inside a process, it draws tool calls and duration as heatmaps on the process diagram, down to the individual tools an agent calls. In the KYC process, that's the difference between knowing the process got more expensive and knowing which agent, and which of its tools, is driving the cost. The optimization effort goes where it pays off: a smaller model or a tighter prompt for the one agent that needs it, instead of a cost cut across every agent.

Did the last change help? Say the team moves the screening agent to a cheaper model. They take a baseline from the control plane first, test the new model with Camunda Process Test, ship it, and compare the weeks after with the weeks before. If the change went out as a new process version, the control plane puts the versions side by side. If the model was swapped at runtime, the shift shows up in the trends from that date on. That comparison catches what a price list hides: a model that's cheaper per token but makes twice the tool calls isn't cheaper per case.

Sometimes the data points at a different change. If the heatmap shows an agent taking the same path on most cases, one or two iterations and the same tool each time, the agent is making a decision the process could make on its own. That path is a candidate for a deterministic step, such as a service task or a DMN decision, which is faster, cheaper, and easier to audit. The agent doesn't have to go away: the process routes the predictable cases down the deterministic path and keeps the agent for the exceptions that need its reasoning. An agentic process is still a process, so you model the change as a new version and migrate the running instances to it. Improving an agentic process sometimes means using less AI, wherever the cases turn out to be predictable.

To see the whole review in one pass, watch the control plane walk through the KYC process in about 90 seconds: the health check, the drift, the spend, and a version change.

Be sure to come back to check out our video about the Optimize Agentic Control Plane (coming soon).

Running agents you can trust, from test to scale

This post closes the Production Agentic Trust series, which started from a simple observation: AI agents don't have a capability problem, they have a trust problem. Across it, the same KYC process has been tested before it reached a customer, watched and explained case by case in Operate, and now measured across hundreds of cases in Optimize. That's what takes an agent from a pilot to production you can defend: you know how it behaves before it ships, what it's doing and why while it runs, and whether it's still worth running months later.

The Agentic control plane is available in Optimize with Camunda 8.10. Pick one agentic process you already run, open it in the control plane, and see whether you can answer all three questions for every agent in it before the next review asks them.