AI investments aren’t like normal tech investments. Most take significantly longer to show a return on investment (ROI). This can make a company’s chief financial officer (CFO) nervous as AI spending climbs.
The delta is measurable. Deloitte’s survey of executives in Europe and the Middle East found that, while the average tech investment sees payoff at the 7-12 month mark, a typical AI use case takes 2 to 4 years. That puts many AI projects at risk of cancellation.
You can’t do much to speed up an AI agent’s payback period. But you can significantly reduce its runtime costs, keeping costs controlled as the agent learns and adapts. One of the best ways is to stop retrieving the context your agents need dynamically, and instead store it directly in your data warehouse.
Our team used this approach to shave more than 50% off token costs. Here’s what we did, why we did it, and why data warehouse context is such a powerful lever for reining in AI costs.
Why AI cost outpaces AI value
Tokens are the basic unit of work in large language models (LLMs). They can represent a word, a part of a word, or a combination of words and punctuation. On top of being the basic unit of LLM work, they’re also the basic unit of costing.
The good news is that, as AI advances, token costs are decreasing rapidly. Adjusted for performance, LLM inference costs drop about 10x a year.
The bad news is that, as with anything compute- and data-related, we use more tokens even as costs plummet. This is especially true of agents, the autonomous software systems at the forefront of modern AI.
An agent consumes more tokens than an operator driving Claude or ChatGPT from their desktop. Single agents typically use about 4x more tokens than a chat interface. A multi-agent system, according to Anthropic, can use about 15x more.
We can see the impact of this expense in IT budgets. Despite falling token costs, experts foresaw a 36% increase in average monthly AI spend in 2025. That trend doesn’t appear to be slowing down in 2026.
When these numbers roll in, they cause CFOs to ask pointed questions like, “Why is our LLM spend so high when our ROI on agentic solutions is so low?” Unfortunately, most companies don’t have a good answer.
This disconnect is causing internal strife. On the one hand, according to a survey of 500 finance leaders at large US and UK enterprises by DoiT, only 15% of companies said they could calculate AI ROI easily. At the same time, 83% of finance leaders expect clear, quantifiable AI results within a year.
How MCP-based context wastes tokens
You can’t easily decrease the time to ROI for AI apps. High upfront integration and production costs, adoption and learning curves, and process re-engineering are all necessary for an agentic solution to succeed in production. These require fundamental changes to your infrastructure and business processes. That doesn’t happen overnight.
If you can’t speed up the process or immediately calculate project ROI, you can at least work on decreasing costs. A major culprit of cost spikes in agentic solutions is the tokens consumed by using Model Context Protocol (MCP).
MCP is often the easiest and fastest way to connect data, since it negotiates capabilities with agents automatically. That prevents data teams from having to wire up dozens of APIs manually to their agents.
That’s why it seemed obvious to us to wire up 20+ MCP servers after rolling out Claude across the company. However, we quickly realized that this was driving up costs in three ways.
Tool definitions on every call
Most MCP clients load every tool definition upfront. That creates a ton of overhead.
Anthropic measured it and found that a five-server setup that uses 58 tools consumes about 55,000 tokens. That’s before the conversation even starts.
At 10,000 agent calls a month, that’s 550 million input tokens of overhead before any work’s done. On Claude Opus 5.5, that comes out to $2,200 a month in pure warm-up fees.
Raw data that flows through the model, sometimes twice
Intermediate results can be passed back and forth in requests and responses. This means the same payload can be counted twice. For example, pulling a sales transcript and then attaching it to a Salesforce lead puts the same data through the model twice, consuming an extra 50,000 tokens per two-hour meeting transcript.
The same context, pulled again and again
In a naive MCP implementation, there’s no reuse. Requesting a chat transcript from a system like Gong or Salesforce pulls the same context every time a request needs the data.
Stitching together raw output from 20+ sources in one chat gets very expensive, very quickly. And none of that work persists between sessions.
How warehouse-first context changes the math
Faced with these inefficiencies, we realized we could eliminate most of them by creating a permanent context layer in our data warehouse. This means that retrieval and data shaping from multiple sources get done once and reused across invocations. If data changes, we implement it as an update to this stored data.
This approach addresses each of the downsides of using straight MCP for context retrieval:
Use fewer tool definitions per call
Instead of querying 20+ MCP servers every time, agents only need to query one governed data warehouse layer. That reduces overhead and decreases request completion time to boot.
This simple change means we don’t suffer the expense of loading tools upfront for every request. Anthropic’s data shows a similar approach .
We leverage the dbt MCP server to make this context available to consumers across the company. We simply skip making tooling calls that carry high overhead.
Use compact context, not raw payloads
With this approach to context, we don’t have to hand off entire payloads (such as the full transcript of a two-hour call) to an agent. We use dbt to chunk, embed, and classify transcripts.
This filters our transcripts down to the most relevant and frequently cited passages. Full transcripts stay in the data warehouse; only the passages a given request or task needs go to the model.
This can net big reductions in token spend. (a 98.7% reduction) by changing how the agent calls MCP tools.
Smaller contexts do more than cut payload costs: they often produce better-quality answers. That cuts down on retries, which reduces response times.
Process context once, then reuse it
Once a piece of context has been extracted and stored, it can be leveraged by every team and agent that needs it. This means teams that need access to critical context don’t need to spend money reinventing the wheel.
Several of our teams need to mine data from customer calls recorded in Gong. Having AI read every call in the dataset costs about $3,000 each time we do it. By storing the most important elements of those calls in the data warehouse, we only need to pull and process new calls as they arrive via Fivetran.
How to build the business case for your CFO
It’s natural that CFOs want to understand the financial impact of the company’s AI projects. Protecting the bottom line is their job.
Your job is to show your investment in the company’s AI future is justified, even in the face of long ROI timelines. This requires showing your project is using its money wisely:
- Establish your baseline spend: tokens, MCP connections, tools loaded, and API costs.
- Find what MCP sources overlap with data already in your warehouse.
- Start with the most expensive, context-rich sources that are currently chewing up the most tokens.
- Use dbt to process this unstructured data into chunks you embed into your data warehouse.
- Measure the before-and-after costs.
With this work in place, you can have a frank and honest discussion with your leadership about agentic AI investments, while showing you’ve done everything to control costs while you’re building your way to returns.
If you find yourself struggling to keep AI workload costs under control, talk to us today about how we can help you create a high-quality, governed, and reusable context layer for your AI agents.
