A while back, I argued that most enterprises don’t have a compute problem; they have a context topology problem. I still think that’s right. What I’ve changed my mind about is when it matters.

I used to treat it as an architecture question: important, but something you could work on in parallel with everything else. After a year of watching companies move AI out of pilots and into production, I’d put it differently. Data context topology doesn’t become the constraint gradually. It becomes the constraint on a specific day, and that day is the day the pilot graduates. By then, it’s the most expensive thing in the building to fix.

The pilot worked. Production is where the shortcuts show

Nearly every enterprise I talk to has a pilot that worked. A team picked a use case, pulled together a data set, cleaned it up, sorted out permissions with a couple of emails and a conversation, and shipped something that impressed the executive staff. That’s a good outcome. It’s how it should go.

The trouble is what the pilot actually proved. It proved the model was capable. It did not prove the organization could do it repeatedly with real, changing data. It’s easy to build a prototype from a copy of data that was hand-selected and cleaned. Lots of people have scars from their big data experiments a decade ago.

Everything that made the AI pilot possible was done by hand, once, by people who already knew where the data lived and what it represented. The lineage was in someone’s head. The access decision was made by a person who understood both systems well enough to make the call. None of that is infrastructure. That works for a pilot, but it doesn’t scale.

So the second team starts over. Then the third. Within a year, you have several teams maintaining several pipelines against overlapping sources, each with a defensible but different view of the same entity, and nobody able to say which one is authoritative.

Compute scales with a purchase order. Knowing which data is right doesn’t, and no purchase order fixes a data estate that was never built for it.

You’re paying compute prices to solve a data problem

Here’s the part I think gets missed, and it’s the reason this keeps surfacing as a compute conversation instead of a data conversation.

Ask an infrastructure team how the AI environment is doing, and you’ll usually get some version of a utilization or consumption metric. In an owned GPU environment, that may be fleet utilization. In the cloud, it may be token usage or inference spend.

But those numbers tell you about the infrastructure being used. They tell you nothing about whether the work is worth doing.

A great deal of it isn’t. The same corpus gets embedded a second time because another team couldn’t find the first team’s vectors or found them and didn’t trust them. The same context gets retrieved and reprocessed on every session because there is nowhere durable to put it. An agent reads the same 40 documents to answer a question it answered yesterday, then does it again tomorrow for the next user. Jobs sit waiting on storage that can’t feed them fast enough, and the waiting counts as busy. None of these scenarios are unusual. They’re the ordinary behavior of a system whose data layer was never designed for reuse.

Now follow that into the AI budget.

Redundant data work does not appear anywhere as a data cost. There is no line item called “context we rebuilt for the fourth time.” It appears as compute pressure: more GPU capacity on premises, more token consumption in the cloud, or both. Queues lengthen, jobs slow down, and the team asks for more GPU capacity, which, given the numbers in front of them, is an entirely rational request.

So you buy the capacity. And that’s the trap. The new hardware absorbs the inefficiency, the pressure disappears for two quarters, and the inefficiency becomes permanent, because nobody audits a system that has stopped complaining. The waste doesn’t get fixed. It gets capitalized.

Agentic workloads only increase that pressure. A single-shot query touches context once. An agent working a multi-step task touches it repeatedly, across steps, across sessions, and increasingly across other agents doing adjacent work on the same underlying sources. The ratio of context handling to actual generation moves in the wrong direction, and it moves as adoption grows, which means the problem is the worst exactly when the system is most successful.

It needs to be acknowledged that some of this is genuinely hard to see. Many organizations cannot tell you their cost per useful token, and the tooling to separate productive AI work from redundant work across an organization is still immature. I don’t think anyone has fully solved that, including us. But hard to measure is not the same as not happening, and the direction of the error is consistent. It tends to overstate how much compute you need.

Governance can either accelerate production or become the bottleneck

Most enterprises think about governance primarily as a compliance requirement. But in production AI, it also has a direct impact on how quickly teams can move.

If a team cannot get a clear answer to “Is this model allowed to use this data?”, the work stops until someone can make that decision. The model may be ready. The data may be available. But if access rules have to be interpreted manually every time, moving from one use case to the next gets slower.

The opposite is also true. When access policies are clear and already built into how data is managed, teams can move faster because they know what they can use without starting a new approval process each time.

Teams move fastest with data that already has clear access rules around it.

I’ll also say plainly that the hardest version of this is unsolved. Controlling which documents a model can retrieve is tractable. Governing what a model may infer from documents it is legitimately allowed to see, and what it may then disclose to a user who shouldn’t have it, is not solved anywhere in the industry, and I include Everpure in that. What you can do today is make it visible: log what agents actually reach for, watch where they get stopped, and use those logs to complement the catalog and show how data is actually being accessed in practice.

3 questions worth asking before expanding your compute

If another team wanted to build on what we’ve already ingested, could they find it, trust it, and use it without rebuilding it?
What share of our inference spend is work we’ve already done?
Can we prove, without convening a meeting, what any given model is permitted to access?

If those answers are uncomfortable, more compute and better models won’t fix the underlying problems. They’ll only make them more expensive.

It’s time to get your data estate in order

The accelerator roadmap is the most interesting it has been in a decade, and I’ll be at NVIDIA GTC Berlin 2026 looking at it with everyone else.

But compute is becoming less of a differentiator on its own. Nearly everyone at that show will have access to broadly the same silicon on broadly the same timeline. What they will not have in common is whether their data is reachable, trustworthy, and reusable at the moment the model needs it.

That’s the harder problem, and it’s the one I want to take up next: what a production AI factory actually requires from the data layer and what it takes to build one that holds up.

I’ll be at GTC Berlin October 20–22, talking with teams about what happens when AI moves from pilot to production. Come find us if you want to compare notes on the data side of that transition.

Check out these resources to learn more: