GTC Berlin made one thing pretty clear: getting an AI factory online is becoming the easier problem.

The conversations I found more interesting were about what happens after that. How do you keep GPUs busy? What happens when multiple workloads start competing for the same infrastructure? How do you deal with failures, power constraints, and all the operational issues that do not show up in a pilot?

That is where things get harder. Infrastructure that can get a model running is not necessarily infrastructure that will behave the same way when hundreds of GPUs become thousands and one workload becomes dozens.

Scale is not the same problem, only bigger

I’ve spent my career building and operating large, distributed systems. The thing that still surprises me is how abruptly scale arrives. It behaves less like a multiplier and more like a phase change.

Every system has a range where its design assumptions hold. Push past that range and problems that looked like edge cases become the dominant engineering effort. Nobody built the system badly. The assumptions underneath it just stopped being true.

Managed service provider STN saw that firsthand. Its previous architecture topped out at around 35GB/s on a highly optimized small file data set, while FlashBlade//EXA™ now delivers up to 100GB/s.

Sabur Mian, CEO and founder of STN, described the scaling challenge this way: “In a typical storage infrastructure, researchers might start training a model on four nodes and get good performance—but as soon as they start scaling up, that performance collapses. With FlashBlade//EXA, we’ve scaled up to 192 nodes so far, and we’ve yet to find the limit.”

A pilot can go from one team to the whole company in two quarters, but the data platform underneath it doesn’t get redesigned in that window. It just gets asked to do 10 times more.

What actually changes on the way up

Four things behave differently at scale than they do in a proof of concept.

Concurrency stops behaving linearly.

A single training job reading a data set is a well-understood I/O pattern. Production is a mix of preprocessing, training and fine-tuning, checkpoint writes, and inference, often running at the same time against the same data. Each has a different access pattern. At scale, the challenge is how those workloads interact when they compete for the same infrastructure.

Most systems hit a metadata wall before a bandwidth wall. Bandwidth charts look fine while workloads get slower, so teams go chasing throughput when the real constraint is how many lookups and creates per second the control path can sustain.

Adding nodes doesn’t fix it because the constraint isn’t in the part you added.

Data movement starts setting the schedule.

In a pilot, moving data is setup work. You stage a data set once and get on with training.

In production, the moving never stops. Checkpoints for large models are enormous, and saving them creates a massive burst of concurrent writes. The write path has to absorb a spike that has almost nothing to do with the average read load you sized around.

Add data set refreshes, replication, and copies made because one team couldn’t use another team’s data in place. All of that adds more data movement at exactly the point when the system is already under pressure. The time your infrastructure spends shuffling bytes instead of computing on them becomes a real line in the capacity plan.

Context outgrows where it was supposed to live.

At small scale, context management can look like an inference problem. At fleet scale, it becomes an infrastructure problem too.

With long-context agentic workloads and thousands of concurrent sessions, managing that context becomes a data-placement problem as well as a GPU-memory problem.

When evicted context has to be recomputed from scratch, you’re paying GPU prices to redo arithmetic you already did. That waste is invisible at a handful of sessions and expensive at thousands.

Failure stops being an event.

At a few racks, hardware failure is something that happens to you. At a few hundred, something is always failing somewhere.

Uptime stops being the useful measure. What matters is how much performance you lose while the system heals and whether a rebuild slows other workloads sharing the system. Two systems can post the same numbers in a bake-off and behave nothing alike once one of them is rebuilding under load.

The economics invert

The budget framing is where I see the most expensive mistakes.

At small scale, the data platform is a cost. You size it, you buy it, and it shows up in the budget as a number you’d prefer were smaller.

At scale, that framing breaks down. When accelerators are most of the capital in the room, the data layer stops being a cost line and becomes a utilization multiplier. Every point of GPU idle time caused by a stall gets charged against the most valuable asset you own.

I see this most clearly with GPU cloud providers, where idle time is directly billable. The math is the same for an enterprise running a large owned fleet. It just doesn’t arrive as an invoice, which makes it easier to ignore.

What this means for how you design

The choices that matter at scale are mostly invisible at pilot size.

Whether metadata scales independently from the data path is not an interesting question at 200 GPUs. Everything works. At several thousand GPUs, mixed tenants, and billions of small files, metadata operations can become the bottleneck even when bandwidth is still available. That means the system can look healthy on a throughput chart while workloads are already starting to slow down. That’s why we built FlashBlade//EXA with a disaggregated metadata core instead of scaling metadata and data together.

The same is true for how applications access data. File and object workloads use different data models, semantics, and access protocols. Storage architectures vary: Some maintain separate namespaces, while others provide cross-protocol access to the same underlying data. FlashBlade//EXA consolidates file and object workloads on a common, high-performance platform while preserving their native access paths and semantics. This allows organizations to simplify infrastructure and operations without compromising on protocol semantics critical for running AI workloads correctly.

Whether the system can isolate tenants determines whether one team’s job ruins another team’s afternoon.

None of these retrofit gracefully. You make them before you need them, which is exactly why they’re easy to skip.

Before you scale, find the constraint

The hard part is knowing which assumption will stop holding first.

Is metadata becoming the bottleneck before bandwidth? Is checkpoint traffic slowing down training or inference workloads on the same infrastructure? Are teams copying data because the system makes reuse difficult? When something fails, does the infrastructure recover without taking the workloads around it down?

Those are not questions you can answer from a pilot.

They require looking at the system as a fleet: how workloads interact, how data moves, how contention shows up, and what happens when the environment is under pressure. That’s the difference between proving that AI can run and knowing that the infrastructure will continue to behave predictably as adoption grows.

Where we take this next

The evidence is the harder part, and it’s what we’re taking to SC26, November 15–20, 2026, in Chicago.

We’ll put those pressures into context: how FlashBlade//EXA handles sustained throughput and metadata concurrency across large, multi-tenant environments, and what S3 over RDMA and MLPerf Storage performance results tell us about checkpointing and KV cache under production-like workloads.

Plenty of people have proven that AI can work. The harder question is what happens when you run it across thousands of GPUs, multiple teams, and production workloads at the same time.

To learn more, check out: