Dev48
Language
  • About
  • Services
  • Industries
  • Technologies
  • Articles
  • Contacts
Book a call
    Home/Articles/Squeezing every drop of inference from your gpus with pure kva
Dev48
info@dev48.org+375 29 320-40-96WhatsApp@isky_by

© 2026 · All rights reserved.

Squeezing Every Drop of Inference from Your GPUs with Pure KVA

Фото: Brett Jordan (Unsplash) — https://unsplash.com/photos/printed-text-in-religious-book-O2mD8qjwswo?utm_source=dev48&utm_medium=referral

Source: Everpure Blog•November 18, 2026 at 03:13 PM

Squeezing Every Drop of Inference from Your GPUs with Pure KVA

Squeezing Every Drop of Inference from Your GPUs with Pure KVA by Everpure Blog Learn how Pure Key-Value Accelerator (KVA) uses a three-tier KV cache to improve GPU utilization and reduce time to first token (TTFT). The post Squeezing Every Drop of Inference from Your GPUs with Pure KVA appeared…

Our pace of innovation around enterprise KV cache continues to impact our customers’ inference stacks with the latest release of Pure Key-Value Accelerator (KVA) adding three-tiered architecture, S3 over RDMA, multi-tenancy, and support for TurboQuant. Today, let’s focus on the new features that impact time to first token (TTFT).

Increase GPU turns per time slice

The new three-tier architecture of Pure KVA (see Figure 1 below) improves KV cache load times and GPU utilization. In short, you get to run more turns in a given span of time on a fixed GPU footprint. This newly released feature enables you to leverage high-bandwidth memory (HBM), system memory, and storage to increase KV cache reuse with faster TTFT.

Figure 1: Three-tiered KV load path for Pure Key-Value Accelerator (KVA).

We tested Kimi-k3 on NVIDIA B300 GPUs, with “vLLM bench” queries of increasing context from 100k tokens to 200k tokens. Here’s an example of the command we used during testing:

vllm bench serve –model google/gemma-4-26B-A4B-it –num-prompts 50 –random-input-len 200000 –max-concurrency 1 –output-len 1

Comparing KV cache load TTFT shows Pure KVA is 5.4X to 11.5X faster than vLLM.

Figure 2: Time to first token running Kimi-k3 on B300 – vLLM vs. Pure KVA (NFS).

Comparing KV cache load TTFT shows Pure KVA is 1.7X to 2X times faster than LMCache. Both Pure KVA and LMCache are defined with a DRAM tier in addition to the storage tier (NFS or FS native).

Figure 3: Time to first token running Kimi-k3 on B300 – LMCache (fs native) vs. Pure KVA (NFS).

Let’s look at a theoretical example with a larger user base: We have eight B300 GPUs to serve 100 users, each with a 256K context, initiating a turn every 60s for 10 turns with 512 input tokens for 1,024 output tokens. Using Pure KVA, we can run more turns per hour (724) than without (313). Our workload finishes 2.3X faster—saving 108 minutes.

Figure 4: Time to complete all sessions—100 sessions each with 10 turns.

The reason is simple: recomputing. Large contexts take longer than reloading from a saved copy. In optimal conditions, a 256k context would take 7.25s to recompute on a B300 GPU. Note that these theoretical numbers, for this example use case, are 2X faster than our real-world results. The three-tiered approach of Pure KVA provides simple KV cache management from HBM to DRAM to storage, so the inference engine can start decoding new tokens instead of recomputing known “old” tokens.

Figure 5: Recompute vs. reload—optimal conditions.*

*Numbers are based on optimal maximum throughputs—example used bandwidth throughput of 55.6 GB/s for DRAM and 10 GB/s.

With our introduction of S3 over RDMA, using object storage on FlashBlade® no longer pays the penalty of the S3 protocol overhead. Our vLLM bench test showed Pure KVA S3 performance was 3.9X to 8.2X times faster than vLLM. This gives you the flexibility to choose the protocol that best suits your inference stack requirements.

Figure 6: Time to first token running Kimi-k3 on B300 – vLLM vs. Pure KVA (S3).

Conclusion

On-premises enterprise inference requires maximizing the GPU footprint you have. The best lever is to reuse KV cache to allow for more user turns per GPU hour. User sessions will churn from the GPU HBM, and recomputing a prior session is a waste of valuable GPU time. Reloading KV cache to eliminate the impact of session churn allows you to squeeze as much inference as possible out of your existing GPU footprint.

Contact your account team today to get faster inference and start saving GPU cycles.

← All articles

What this article says

More in Hardware & Electronics

All →
Global research from Schneider Electric finds demand for on-prem AI infrastructure is surging due to sovereignty and compliance concernsGlobal research from Schneider Electric finds demand for on-prem AI infrastructure is surging due to sovereignty and compliance concerns
Schneider Electric

Global research from Schneider Electric finds demand for on-prem AI infrastructure is surging due to sovereignty and compliance concernsGlobal research from Schneider Electric finds demand for on-prem AI infrastructure is surging due to sovereignty and compliance concerns

GE HealthCare brings advanced visualization into <a name="OLE_LINK8" id="OLE_LINK8"></a>InteleViewer to support radiology workflows
Ge Healthcare

GE HealthCare brings advanced visualization into <a name="OLE_LINK8" id="OLE_LINK8"></a>InteleViewer to support radiology workflows

Bring sovereign, secure AI to where your business runs
Lenovo

Bring sovereign, secure AI to where your business runs

NVIDIA RTX Spark Compact Desktops Out Now
NVIDIA

NVIDIA RTX Spark Compact Desktops Out Now

Table ronde : grand renouvellement des réseaux : 2026 allume la mèche, 2027 fera t-il exploser le marché ?
TP-Link

Table ronde : grand renouvellement des réseaux : 2026 allume la mèche, 2027 fera t-il exploser le marché ?

ABB cares for healthcare: Top three technologies | ABB
Abb Energy

ABB cares for healthcare: Top three technologies | ABB

More from Pure Storage

What Production AI Demands from the Data Layer
Pure Storage

What Production AI Demands from the Data Layer

PX-CSI Support for Talos Linux Coming in December
Pure Storage

PX-CSI Support for Talos Linux Coming in December

The Data Foundation behind Frontier Transformation
Pure Storage

The Data Foundation behind Frontier Transformation

You’re Buying Frontier Models for Economy Tasks. Benchmark Your Coding Agents on Your Own Code
Pure Storage

You’re Buying Frontier Models for Economy Tasks. Benchmark Your Coding Agents on Your Own Code

A vSphere Admin’s Day-Two Translation Guide to VMs on Kubernetes
Pure Storage

A vSphere Admin’s Day-Two Translation Guide to VMs on Kubernetes

Portworx Unveils Advanced AI and VM Capabilities on Kubernetes; Everpure Named a Leader in 2026 IDC MarketScape Report
Pure Storage

Portworx Unveils Advanced AI and VM Capabilities on Kubernetes; Everpure Named a Leader in 2026 IDC MarketScape Report