By now, the token spend problem is well documented. And it’s not going away: Goldman Sachs expects token usage to grow ~24x by 2030.
That means AI leaders are tackling different questions. We’ve noticed this in our own conversations. We no longer hear, “Is my agent good enough?” Now, the question is: “Can we afford to run it at scale?”
This shift has created new technical challenges. Teams need automated systems that cut wasted tokens without hurting agent quality – and keep improving those cuts as models, prices, and workflows change.
We help companies reduce token spend without sacrificing quality. Apply for early access here
It’s not just headlines
We spoke to one CTO of an enterprise AI coding company, who succinctly captured this moment: “I understand you spent $1M in tokens, but what was the impact? No one can answer that.”
Teams know they need to cut costs. But two challenges are getting in their way. Across dozens of private conversations with teams running agents in production, this is what we heard:
- Manually tuning agents is slow and brittle: New optimization methods ship weekly, and not every trick works. A VP of AI at an S&P 500 software company told us how their routing rules that shift traffic to cheaper models eventually drove cost up because the weaker model produced dramatically longer trajectories. Landing on the right logic for every use case can take weeks of experimentation.
- Model changes press reset: Agent optimization doesn’t come with a lifetime guarantee. A CEO at an AI coding unicorn shared how their pricing and performance assumptions get thrown out every time their underlying model ships changes. Same thing for adding a new model, tools, or adjusting the harness: The ground shifts, and carefully tuned agents or routing rules go stale.
The picture we got from the field is that agent optimization – cutting cost without harming quality – remains hand-authored and imprecise per use case; getting around those barriers requires a level of resources, time, and expertise that many companies don’t have at their disposal.
So what are teams doing in the meantime?
Teams are converging on routing
You can squeeze cost all over the stack, but the move we hear most is routing: send each call to the cheapest model that can handle it. Routing is everywhere:
- From industry leaders: Just in the past month alone, leaders such as Salesforce CEO Marc Benioff, Harvey Co-Founder Gabe Pereyra, or Box CEO Aaron Levie have all made the case for routing.
- In the production data: Vercel’s new report found that teams operating at scale run an average of 35 models as a routing graph. This is now production best practice.
- In our own lab, too: We’ve spent time researching the benefits of diverse model and agent ensembles, eventually leading us to reach #1 on BrowseComp-Plus and DeepResearch Bench II over the last few weeks..
The hard part is making routing automatic, with logic that learns the best calls per agent and per domain. It has to account for things like provider specificities, KV caching, query differences and more.
How we’re automating agent optimization
We’re starting by building an intelligent router, a drop-in layer that you point your agent endpoint at to find and cut waste at runtime. At the base, it spots the usual suspects for token leakage (e.g. redundant context, duplicated instructions, irrelevant tools) and trims them, in a cache-aware and consistent way. Above that, it makes advanced routing decisions that keep improving over time, as traffic accumulates.
It’s working: Routing a popular code-review agent this way cut cost per PR by 68% against public baselines. That’s about $560K a year on production traffic, from a single agent. (More on that research soon.)
We’re not stopping there. Over the past 6 months we’ve shown new state-of-the-art and cost-efficiency on 4 different benchmarks, by applying agent-specific optimization methods that go beyond routing and waste-trimming.
The instinct is the same, whether we’re optimizing at the routing point or digging into the agent architecture: cut costs, don’t break what works, and keep learning. By automating that loop, we’re helping more teams confidently answer that question: Can we afford to run it at scale?
If this is a problem your team is living, let’s talk. Apply for early access to our intelligent router →
