Dev48
Language
  • About
  • Services
  • Industries
  • Technologies
  • Articles
  • Contacts
Book a call
    Home/Articles/The air cooled token factory intel gpu d series sambarack sn50
Dev48
info@dev48.org+375 29 320-40-96WhatsApp@isky_by

© 2026 · All rights reserved.

The Air-Cooled Token Factory: Intel GPU D-Series + SambaRack SN50

Source: SambaNova•November 16, 2026 at 05:52 AM

The Air-Cooled Token Factory: Intel GPU D-Series + SambaRack SN50

Over the last several months, many customers have been evaluating and deploying disaggregated inference workloads to help optimize the speed and economics of AI. At SuperComputing 2026, SambaNova and Intel are excited to demonstrate how we are advancing a heterogeneous inference architecture…

Over the last several months, many customers have been evaluating and deploying disaggregated inference workloads to help optimize the speed and economics of AI. At SuperComputing 2026, SambaNova and Intel are excited to demonstrate how we are advancing a heterogeneous inference architecture that pairs Intel GPU D-Series for prefill with SambaNova SN50 RDUs for decode. The end result is a solution that delivers an air-cooled Token Factory, so that each phase of inference runs on the hardware best suited to it. Below is a recorded demonstration of the first system running GLM-5.3 on both hardware stacks in an air-cooled facility. As we will explore in this blog, customers will be able to deploy this solution in existing brownfield data centers, which will become increasingly important over the next couple years.

Intel’s Air-Cooled Prefill Engine

Introduced at OCP, the D-Series, codenamed Crescent Island, is a 350 W air-cooled PCIe GPU for data center inference. In a disaggregated serving system, it serves as a dedicated prefill engine for long-context workloads. Its 32 Xe3P cores and 256 XMX engines provide the compute for processing prompts and building the KV cache. 160 GB of LPDDR5X on the Intel card, and up to 480 GB in partner designs, provide room for model weights and growing KV caches, while 1.5 TB/s of memory bandwidth helps move data during prefill. See Intel’s product page for GPU specifications and the OCP presentation for the broader system design.

Intel offers the D-Series in multiple rack configurations. The reference design shown at OCP holds 80 D-Series GPUs and delivers 208 petaflops of peak FP4 compute, so a single rack can process large volumes of long prompts in parallel. Memory ranges from 12 TB to 38 TB per rack depending on configuration, with 120 TB/s of aggregate memory bandwidth to keep KV cache construction moving as context windows grow. Other configurations can vary GPU count, memory, and power to fit a provider’s facility and workload mix. Paired with SambaRack SN50 for decode, this gives providers a prefill tier and a decode tier that each scale to match their traffic. The SuperComputing 2026 demonstration runs on a smaller configuration: four D-Series GPUs for prefill, paired with SambaRack SN50 for decode.

SambaNova’s Air-Cooled Decode Engine

SN50 is SambaNova’s fifth-generation Reconfigurable Dataflow Unit (RDU), built for high-performance and energy efficient inference for the decode phase. Unlike GPUs which run a model as a series of separate kernels, SN50 RDUs chain them into a continuous dataflow on and across chips instead of sending data back to memory between operations, which cuts memory traffic and kernel relaunch overhead during token generation. Each SambaRack SN50 holds 16 RDUs with a typical 20 kW of power required for inference, allowing it to fit in many air-cooled data centers. For larger models , 16 SambaRacks work together in one scalable unit of up to 256 RDUs, A single workload spreads across many lower-power racks instead of one dense, liquid-cooled rack. This increases tokens per second per chip for medium-batch workloads, while keeping per-request speed high. Earlier this year, SambaNova showed SN50 decode with NVIDIA H200 prefill running MiniMax M2.7, as benchmarked by Artificial Analysis, and SemiAnalysis benchmarked SambaRack SN50 on the same model.

The Blueprint for Air-Cooled Data Centers

For neoclouds and inference providers, hardware only pays off if the service feels fast to each customer and keeps enough capacity busy to earn back the investment. That balance is hardest with frontier models serving agentic workloads. The D-Series GPU and SambaRack SN50 split the work by phase: the GPU runs compute-heavy prefill, building the KV cache from the incoming prompt, and SN50 runs memory-bound decode, generating output tokens one at a time. What makes the two behave like one service is the software that runs across them.

The demonstration runs on an open serving stack many providers already use. NVIDIA Dynamo sits in front of both pools and is designed to route each request to where its context is already cached. vLLM serves the prefill path on the GPU and the decode path on SN50, and NIXL moves the KV cache between them over UCX/RDMA on standard RoCEv2 Ethernet. For a provider, that means teams work with serving tools they already know instead of learning a new stack, and the KV handoff runs over the Ethernet fabric already in the data center rather than a proprietary interconnect.

Cache-aware routing is where much of the efficiency comes from. SN50 keeps KV prefixes from earlier prompts in its attached DDR, alongside on-chip SRAM and HBM for the active decode path. When an agent returns to a conversation or revisits shared context, Dynamo is designed to send the request to the SN50 that already holds that prefix, so the GPU only processes the new context and less KV state crosses the network. Each cache hit saves the GPU from repeated prefill work, which shortens the wait for the first token and frees GPU capacity for new requests.

Because prefill and decode run as separate pools, each one scales on its own. A provider whose traffic shifts toward long prompts can add D-Series GPUs, and one whose agents generate long outputs can add SN50 capacity, without resizing the other side. SambaNova’s Hot Chips blog goes deeper on the dataflow architecture and Model Bandwidth Utilization behind decode, and another SambaNova blog explains why DDR matters for the cache.

Why existing air-cooled sites matter

As many as half of the data center projects worldwide slated to come online in 2026 could face delays, held back by power constraints, grid equipment shortages, and growing community opposition, according to Sightline Climate (reported by Axios, February 2026). That makes the space providers already run more valuable than the space they’re waiting on. Much of that existing space is air-cooled, and many of the newest AI systems need direct liquid cooling, which means a costly, time-consuming retrofit before they can be installed.

By orchestrating workloads across Intel GPU D-series and SambaNova RDUs, this heterogeneous approach represents the new baseline for the fast inference required for agentic workflows and keeping power demands safely within existing air-cooled rack limits. Intel and SambaNova are building this solution to help providers grow supply to meet AI token demand without waiting for new facilities and utilizing the most common software stacks.

If you’re at SuperComputing 2026, come see the systems at SambaNova booth 2246, visit Intel at booth 2819, and talk with us about how it could fit your data center.

← All articles

What this article says

More in AI & Machine Learning

All →
Apple launches new app to study how people use AI for healthПресса
Apple

Apple launches new app to study how people use AI for health

The best way to monitor your Code Reader from your office
Keyence

The best way to monitor your Code Reader from your office

WHY doesn't MY CODE READER WORK?
Keyence

WHY doesn't MY CODE READER WORK?

The Basics of Wire Simplification with PoE
Keyence

The Basics of Wire Simplification with PoE

Apple launches research study to explore the use of AI for health
Apple

Apple launches research study to explore the use of AI for health

AI Day Seoul Brings Together Korea’s Industry Leaders, Startups and Developer Communities
NVIDIA

AI Day Seoul Brings Together Korea’s Industry Leaders, Startups and Developer Communities

More from SambaNova

GLM-5.3 Now Running on SambaCloud
SambaNova

GLM-5.3 Now Running on SambaCloud

Supermicro Will Build and Sell SambaNova-Powered Inference Systems
SambaNova

Supermicro Will Build and Sell SambaNova-Powered Inference Systems

MiniMax M3.1 Flash Now Available on SambaCloud
SambaNova

MiniMax M3.1 Flash Now Available on SambaCloud

What Is an AI Factory? Definition & Architecture
SambaNova

What Is an AI Factory? Definition & Architecture

MiniMax M3 Prompt Caching on SambaCloud: How It Works
SambaNova

MiniMax M3 Prompt Caching on SambaCloud: How It Works

Sovereign AI: Own Your Infrastructure, Models & Inference
SambaNova

Sovereign AI: Own Your Infrastructure, Models & Inference