Dev48
Language
  • About
  • Services
  • Industries
  • Technologies
  • Articles
  • Contacts
Book a call
    Home/Articles/What production ai demands from the data layer
Dev48
info@dev48.org+375 29 320-40-96WhatsApp@isky_by

© 2026 · All rights reserved.

What Production AI Demands from the Data Layer

Source: Everpure Blog•November 17, 2026 at 03:01 PM

What Production AI Demands from the Data Layer

What Production AI Demands from the Data Layer by Everpure Blog Explore how a scalable AI data layer can accelerate checkpointing and inference while helping keep GPUs productive across AI workloads. The post What Production AI Demands from the Data Layer appeared first on Everpure Blog .

For most of my career, I have sized storage performance by the terabyte.

That is not a figure of speech. In the cloud services I worked on before joining Everpure, you bought throughput in fixed ratios against capacity: a gigabit per second for every terabyte provisioned, or a tier that gave you 125, 250, 500, or 1,000 MBps per provisioned TB. It’s a clean model and an easy one to price. It also means that when a training job needs bandwidth rather than space, you buy space you will never fill to get the bandwidth that is actually keeping your accelerators busy. The economic impact often shows up as underutilized GPU capacity rather than as an obvious storage cost, which is why it tends to go unnoticed.

That is the first example of a pattern I noticed when I joined the FlashBlade//EXA™ team two months ago. In his blog post, “Production AI Doesn’t Just Need to Work. It Needs to Scale,” Cameron Bahar, General Manager of the FlashBlade//EXA business, argued that production AI at scale is a different engineering problem rather than a bigger version of the same one. I’d narrow that further: Many of the problems that appear at scale come from two things that grow at different rates sharing one resource. Control path and data path. Throughput and capacity. Context and GPU memory. One tenant and the next.

So the useful question is not just how fast the storage is. It’s what each of those couplings costs you in AI terms, and what changes when you separate them.

The bottleneck that does not appear on a throughput chart

One of the easiest failure modes to misdiagnose is metadata saturation.

Lookups, creates, stats, and deletes run through a control path. In most scale-out architectures, that control path shares controllers, CPUs, and network ports with the payload, so the two scale together at whatever ratio the hardware imposes. When a workload is metadata-heavy, as many preprocessing pipelines, large RAG corpora, and inference jobs can be, the control path can saturate first. Bandwidth utilization still looks comfortable. Epoch times get longer. Nothing on the storage dashboard explains why.

FlashBlade//EXA runs those paths on different hardware. Metadata sits on the FlashBlade® metadata core, while data sits on standard Linux nodes with NVMe behind them. A client negotiates file layout with the metadata core, then reads and writes directly to the data nodes over RDMA. That means teams can scale metadata performance and data throughput independently, rather than overprovisioning one just to get more of the other. FlashBlade//EXA handles up to 4.6 billion metadata operations per second.

Figure 1: FlashBlade//EXA separates the metadata and data paths so each can scale independently based on workload demand.

Because the metadata and data paths scale independently, teams can add performance where they actually need it. If small-file concurrency is the problem, they can scale metadata without adding throughput or capacity they do not need. And because payload data does not transit a central controller, performance scales with the fabric and data nodes rather than being capped by a shared control point.

STN, a GPU cloud provider running large-scale AI training and inference workloads for multiple customers, is the clearest evidence I have that this holds outside a lab. Its previous architecture topped out at around 35 GB/s on a highly optimized small-file data set, while FlashBlade//EXA now delivers up to 100 GB/s. Sabur Mian, CEO and founder of STN, described the scaling challenge this way: “In a typical storage infrastructure, researchers might start training a model on four nodes and get good performance—but as soon as they start scaling up, that performance collapses. With FlashBlade//EXA, we’ve scaled up to 192 nodes so far, and we’ve yet to find the limit.” The point is not the peak figure, now up to 100 GB/s. It’s that growth stopped being an architectural event.

None of this needs a proprietary filesystem client, so there is no kernel module to patch across several hundred GPU nodes on every upgrade. Having operated a managed Lustre service, I read that as engineering hours returned to the AI team rather than as a storage feature.

Why checkpointing matters at AI scale

Checkpointing is one of the places where storage performance shows up directly in GPU productivity.

During large distributed training, much of the steady-state I/O is read-heavy. A checkpoint reverses that pattern abruptly: Every rank writes its shard at once, creating a burst of write traffic that looks very different from the read load the system handles during training. While that burst drains, the job is not training. Every accelerator in it is waiting, and the length of the wait is set by the write path.

MLPerf Storage v3.0 is the first version of the benchmark that tests this directly because MLCommons added checkpointing and KV cache workloads built from the I/O behavior of large-scale training and inference.

FlashBlade//EXA ranked #1 on every checkpointing and KV cache metric we submitted in the MLPerf Storage CLOSED division. With 1,024 simulated accelerators and a 1.25T-parameter model, it wrote the checkpoint at 877.52 GiB/s, finishing in 17.74 seconds, and read it back at 588.28 GiB/s in 28.99 seconds.

Figure 2: MLPerf Storage v3.0 results show FlashBlade//EXA leading across the submitted checkpointing and KV cache metrics.

The scaling matters as much as the peak result. With 10 data nodes, checkpoint write bandwidth was 327.65 GiB/s. With 30 nodes, it reached 877.52 GiB/s. On the smaller configuration, the same checkpoint would keep all 1,024 simulated accelerators waiting roughly 2.7 times longer. That is why checkpoint performance is not just a storage metric. It affects how much time the AI system spends training versus waiting.

Those results are specific to checkpointing and KV cache workloads. They’re useful proxies for the write burst and context path, but they are not a substitute for testing the actual mix of preprocessing, training, and inference workloads in your environment.

Data movement should not require another infrastructure tier

A large share of production AI data movement is not required by the workload itself. It exists because file data and object data live in separate systems, so someone copies a data set to make it usable by the next stage. Each copy costs capacity, creates a freshness problem, and adds load to a system already under pressure. None of it moves a model forward.

FlashBlade//EXA consolidates file and object workloads on a common, high-performance platform while preserving their native access paths and semantics. More importantly for AI, it can deliver high-performance object access directly, without requiring teams to stand up a separate file cache tier or modify the client stack just to get data to GPUs faster.

Purity//FB 4.8.4 added S3 over RDMA to FlashBlade//EXA, moving object payload over the same RDMA data path used for high-performance file access, with standard S3 API calls and no client-side kernel changes. It’s still a newer capability, with early-release constraints customers should review before designing around it.

What that removes matters more than any throughput figure. A common pattern for feeding GPUs from object storage is to put a fast file cache tier in front of it, then run two systems and keep them coherent. Direct, high-performance object access can reduce the need for that intermediate tier, along with the copy and operational overhead that comes with it.

Recomputation is the most expensive habit in inference

This is where the data layer stops being adjacent to inference economics and becomes part of them.

When context is evicted and has to be rebuilt, the accelerator redoes the arithmetic it already completed. At a handful of sessions, that is invisible. Across thousands of concurrent agentic sessions with long context, it becomes a standing tax on the most expensive hardware in the building. It’s also paid in the metric users feel first, which is time to first token.

Pure Key-Value Accelerator (KVA) persists attention states between sessions rather than discarding them and integrates natively with vLLM. In our benchmarking, it delivered up to 20X faster inference on NFS and 6X on S3, with time to first token two to three times better than other KV caching approaches. FlashBlade//EXA is the tier underneath it once that cache outgrows GPU memory.

MLPerf measured the storage side independently: 85,736 tokens per second on the Llama 3.1 8B storage-only workload, 67,642 on the 8B workload using storage plus memory, and 33,403 on the 70B storage-only workload.

Storage-only is the case I would look at hardest. It means the cache was served entirely from FlashBlade//EXA with no help from host memory and still sustained tens of thousands of tokens per second. That is the difference between persistent context being an architecture you can plan around and a demo you can show.

Isolation is how predictability survives multiple tenants

Multi-tenancy turns predictable performance into a customer commitment. The stakes are concrete, because an inference SLA is a latency promise and a training schedule is a delivery date. Both depend on performance a tenant cannot see or control.

The usual answer is QoS policy. Policy helps. It also runs on shared controllers, which means a burst gets arbitrated at a point both tenants depend on.

FlashBlade//EXA assigns each file system to a data node group, a defined subset of data nodes. A tenant’s data lands on those nodes and nowhere else, so two tenants in different groups are not competing for the same drives or ports. The noisy neighbor problem is absent by construction rather than managed down. The same boundary bounds failure: A data node going down does not touch file systems in other groups. For anyone running AI as a service, that boundary is what makes a performance commitment to one customer survive the arrival of the next one.

Measure the AI system, not the array

The point of separating all of these resources is not a better storage score. It’s a more predictable AI system. The tests I would run are AI-system tests, and none of them produce a storage number.

Measure goodput, the amount of useful work completed, not just GPU utilization. A busy accelerator is not necessarily a productive one. Time a full checkpoint write at the parallelism you expect next year, then multiply the stall by your checkpoint frequency and your accelerator count. Watch time to first token under real concurrency, with and without cache reuse, and find out what share of your prefill work is repeat work. Count the copies of your largest data set and establish what each one exists to work around. Fail a node under load and measure what the workloads beside it lose while the system heals. If power and footprint are constraints where you operate, measure watts per sustained GB/s rather than watts per terabyte—throughput is what feeds GPUs, not capacity.

None of those questions can be answered from a pilot, and a peak throughput figure does not answer any of them.

Come find us at SC26. Bring the configuration you’re worried about, and we’ll walk through how it behaves under the workloads that matter to you.

← All articles

What this article says

More in Hardware & Electronics

All →
The infrastructure behind a technology first, AI-ready venue
Cisco

The infrastructure behind a technology first, AI-ready venue

„Das ist Wireless ganz anders gedacht!“
Sennheiser

„Das ist Wireless ganz anders gedacht!“

Real-Time AI Inferencing Demands a Distributed Front-End Network
Arista Networks

Real-Time AI Inferencing Demands a Distributed Front-End Network

Arduino is heading to Maker Faire Shenzhen 2026
Arduino

Arduino is heading to Maker Faire Shenzhen 2026

Panasonic Announces the LUMIX SHX1, the First Compact Cinema Camera in LUMIX History
Panasonic

Panasonic Announces the LUMIX SHX1, the First Compact Cinema Camera in LUMIX History

GE HealthCare?s Critical Care Suite 3.0 brings first on-device AI for feeding tube detection to X-ray imaging in adult and pediatric care
Ge Healthcare

GE HealthCare?s Critical Care Suite 3.0 brings first on-device AI for feeding tube detection to X-ray imaging in adult and pediatric care

More from Pure Storage

PX-CSI Support for Talos Linux Coming in December
Pure Storage

PX-CSI Support for Talos Linux Coming in December

The Data Foundation behind Frontier Transformation
Pure Storage

The Data Foundation behind Frontier Transformation

You’re Buying Frontier Models for Economy Tasks. Benchmark Your Coding Agents on Your Own Code
Pure Storage

You’re Buying Frontier Models for Economy Tasks. Benchmark Your Coding Agents on Your Own Code

A vSphere Admin’s Day-Two Translation Guide to VMs on Kubernetes
Pure Storage

A vSphere Admin’s Day-Two Translation Guide to VMs on Kubernetes

Portworx Unveils Advanced AI and VM Capabilities on Kubernetes; Everpure Named a Leader in 2026 IDC MarketScape Report
Pure Storage

Portworx Unveils Advanced AI and VM Capabilities on Kubernetes; Everpure Named a Leader in 2026 IDC MarketScape Report

Veeam and Everpure Help Restore the Business
Pure Storage

Veeam and Everpure Help Restore the Business