For service providers, AI creates a familiar strategic question. Will the network carry someone else’s value, or will it become the platform on which that value is created? Inference gives service providers a strong opportunity to choose the second path. They already have assets that the AI economy needs: distributed facilities, power, trusted infrastructure, and reach in the regions where enterprises and governments operate.

Those assets become more valuable when they can operate as one system. An inference request may have suitable capacity near by, yet still travel farther than necessary because the network does not have enough context about the workload. The best destination may depend on latency, available throughput, cost, or a jurisdictional requirement. Conventional reachability alone does not express all those needs.

At NVIDIA GTC Berlin, Cisco is introducing a blueprint for AI Inference Routing, one of several areas where Cisco and NVIDIA are working together for service providers. It outlines how service providers can place, route, secure, and assure inference across a distributed footprint, from a single edge site to a sovereign regional hub. The goal is to solve utilization through placement and routing rather than more infrastructure. Capacity sitting idle in one region can be applied to demand in another, and the resulting service can be built on demand rather than pre-provisioned site by site.

Why inference changes the role of the network

An AI prompt may look like a simple exchange between a user and a model, but the underlying transaction crosses a distributed system. It can pass through the Internet or an enterprise network, policy engines, API gateways, model-routing layers, CPU serving infrastructure, and a GPU fabric before the response returns. That makes placement and connectivity part of the application experience.

This matters even more as inference spreads across central AI factories, regional data centers, metro locations, and edge sites. One request may need the lowest-delay path. Another may need to remain within a national boundary. A third may prioritize cost while still meeting a performance objective. Service providers need a way to translate those requirements into network behavior, and to verify with measurement that the service is delivering what was committed.

A blueprint built on deployed network standards

The AI Inference Routing blueprint starts with mechanisms that service providers already use, including Segment Routing over IPv6 (SRv6), Flexible Algorithm (Flex Algo), and Border Gateway Protocol (BGP). A customer states what they need, such as response time, jurisdiction, resiliency, or cost tier, and the blueprint describes how the network can interpret that intent to make inference locations reachable and select paths that satisfy it. It does not require a new routing protocol or a separate network for every workload.

Flex Algo can calculate paths using defined metrics and constraints. Segment Routing can steer traffic along those paths, while BGP distributes reachability information across the footprint. Used together, these mechanisms provide a practical foundation for routing an inference request toward an eligible location and across a path that reflects its latency, throughput, cost, and jurisdiction requirements. In this design, paths can be instantiated when a service requires them rather than pre-built between every pair of locations, and network slicing can keep inference traffic isolated from other services on the same converged fabric.

For service providers, the business value is straightforward: distributed compute no longer has to be sold as a collection of isolated sites. It can become a service with consistent policy and assurance across a broader footprint. That creates a foundation for differentiated inference connectivity, sovereign AI services, and managed edge AI offerings.

Resilience, security, and operations must work together

Delivering that service consistently across a complex, distributed footprint means these pieces can’t operate in isolation. The blueprint brings them together as one coordinated system.

Resilient infrastructure

The blueprint is designed to treat every participating inference site as part of one reachable system. Policies can distinguish among service objectives instead of sending every request along the same default path.

Because the design uses established transport mechanisms, service providers can build on their current network architecture and operating knowledge. The blueprint also defines how to measure results by verifying response time, the path taken, and the location that served the request against what the service provider committed. Service providers can then prove that they met each service objective.

Embedded security

Distributed inference increases the number of network, cloud, and edge domains that a workload may cross. Security and sovereignty need to be considered with path selection. The blueprint places policy within the routing decision, showing how service providers can keep traffic on approved paths and apply protection consistently across access, metro, core, and edge domains.

Agentic operations

Service providers and large enterprises are balancing capability against control as they bring AI into network operations. Open models deliver on both, giving customers the control and economics that they want without sacrificing the capabilities that they need.

Cisco’s portfolio already includes elements that service providers can use to execute this blueprint. For example, Crosswork AI, Cisco’s multi-agentic framework for network operations, integrates NVIDIA Nemotron open models and NVIDIA AI Enterprise software to support agentic tasks, such as deep network troubleshooting across a distributed footprint with the option to run models on premises.

Crosswork AI observes network conditions, correlates signals across domains, and proposes a resolution before an issue affects service. Service providers can also build their own agents for their environments. Policy checks and operator approval remain part of the workflow, providing a governed path toward greater autonomy.

Turning network reach into new services

Service providers have spent decades building resilient networks, regional facilities, and operational expertise. AI inference gives those investments new roles. Following this blueprint, an operator can use itto build toward offering customers:

Inference connectivity services that select paths for defined latency, throughput, cost, or policy requirements
Sovereign AI services that direct traffic through approved infrastructure and within required boundaries
Managed distributed inference that combines regional or edge capacity into one service experience

Service providers can shift from selling undifferentiated bandwidth to delivering an assured outcome. Customers do not have to assemble the network, compute locations, policy controls, and operating model themselves. The service provider can bring those elements together and manage them as one service.

Extending our work with NVIDIA

The AI Inference Routing blueprint is part of a broader effort between Cisco and NVIDIA to help customers build and operate AI from core to edge. Cisco Secure AI Factory with NVIDIA provides a modular, pre-validated architecture for the AI workloads that run on the infrastructure. The blueprint extends that work to how requests reach distributed capacity across the provider network. Together, these efforts connect the AI stack with the network that carries its traffic.

The infrastructure to compete in distributed AI is already present in many service provider networks. Our goal now is to help service providers make those assets function as a coordinated system, with the resilience, security, and operational control required for carrier-grade services