Self-Hosted LLM vs API: When Does Self-Hosting Pay Off

If you pay an API bill for an open-weight model, you may wonder whether your own GPU would cost less. GPU rent is charged every hour, busy or idle, while an API charges only for the tokens you use.

A self-hosted LLM costs less only when your traffic keeps the GPU busy enough of the time. At October 8, 2026 prices, a 27B model on one rented RTX 3090 breaks even at about 12% load, while a large mixture-of-experts model on rented H200s never beats the cheapest zero-data-retention (ZDR) API endpoint.

This post works out the break-even with live per-provider prices from our API, then shows how our data controls cover privacy.

TL;DR

• Break-even load is the GPU’s hourly rent divided by what the same hour of tokens would cost through the API. Below that load, the API is cheaper.

Break-even load is the GPU’s hourly rent divided by what the same hour of tokens would cost through the API. Below that load, the API is cheaper.

• DeepSeek V4.1 Flash on four rented H200s never breaks even against the cheapest ZDR endpoint. On two B200s, it needs 76% load at the lowest rent we found.

DeepSeek V4.1 Flash on four rented H200s never breaks even against the cheapest ZDR endpoint. On two B200s, it needs 76% load at the lowest rent we found.

• At October 8 prices, Qwen3.8 27B on one RTX 3090 at $0.50 an hour breaks even at 6 to 12% load, but each user gets only about 15 tokens per second.

At October 8 prices, Qwen3.8 27B on one RTX 3090 at $0.50 an hour breaks even at 6 to 12% load, but each user gets only about 15 tokens per second.

• Published self-hosting costs of around a cent per million tokens mostly count cached input tokens, which are nearly free through the API too.

Published self-hosting costs of around a cent per million tokens mostly count cached input tokens, which are nearly free through the API too.

• Most privacy requirements don’t need self-hosting. provider.zdr limits a request to zero-data-retention endpoints, and our EU and US domains keep requests in-region.

Most privacy requirements don’t need self-hosting. provider.zdr limits a request to zero-data-retention endpoints, and our EU and US domains keep requests in-region.

How do you work out whether self-hosting pays off?

Take one hour of a GPU’s time and price it two ways. On one side is the rent. On the other is what the API would charge for the tokens that GPU can serve in an hour, at a speed your users accept.

A break-even of 30% means the GPU has to be busy for 30% of the hours you rent it before it costs less than the API. Anything above 100% means the API wins even if the GPU never rests.

Both sides have to be priced on the same token mix, because input, cached input and output tokens have very different API prices. We also left out engineering time and the spare capacity you’d keep for traffic peaks, so every break-even in this post is a lower bound.

What does the API cost for the same open-weight model?

Most open-weight models are served by many providers. Each one runs its own copy of the weights on its own hardware and sets its own price. Our endpoints API (GET /api/v1/models/{author}/{slug}/endpoints) returns every provider’s price and precision for a model, and GET /api/v1/endpoints/zdr lists the endpoints with a ZDR policy. As of October 8, 2026, they returned:

For the same weights, the most expensive provider charges 3.5 times the cheapest for Qwen3.8 27B output, and nearly 13 times for DeepSeek V4.1 Flash. A break-even worked out against a single list price can be far off. Providers also serve different precisions, from fp4 to bf16, and our LLM quantization post measures how much that changes the answers.

What we paid for real workloads

Reasoning models spend output tokens thinking before they answer, and agents resend their whole context every turn. So we ran two workloads per model with ZDR enforced and priced each request with GET /api/v1/generation, which reports the tokens and cost the provider actually billed:

The last column divides the full cost, input included, by the output tokens. Almost every output token was reasoning, and a self-hosted copy of the model would spend the same GPU time producing them. Qwen3.8 27B also hit our 16,000-token cap while still thinking on 3 of its 7 agent turns. Every API run for this post cost $0.49 in total.

What does a self-hosted LLM cost to run?

We didn’t rent these GPUs ourselves. The self-hosted side uses published benchmarks and rental prices, so treat each row as an assumption:

DeepSeek V4.1 Flash has 552B backbone parameters, so it needs a multi-GPU node. Qwen3.8 27B is a dense model that fits on one 24 GB card at 4 bits.

Throughput is per GPU, the way InferenceX reports it, so every cost in this post is per GPU-hour. Per-user speed counts only the time a response is being generated, and these agentic sessions spend much of their time on prefill, so it doesn’t multiply out to the total. The 3090 figure comes from HyperQwen’s heavily patched vLLM build, so treat it as a best case.

We used its end-to-end figure, which includes prefill, rather than its 1,035 tok/s decode-only headline. HyperQwen benchmarked 128-token prompts, and our chat prompts are much longer, so for our chat mix we estimated 565 output tokens per second from its measured prefill and decode rates.

When does self-hosting pay off?

For each setup, we took the hour of tokens the benchmark measured, split it into uncached input, cached input and output, and priced that mix at every ZDR endpoint for the model, then divided the rent by the result. API prices are from October 8, 2026:

“Never” means the rent is higher than the API cost even at 100% load. Prices move quickly. On October 19, a new ZDR endpoint for Qwen3.8 27B charged $0.50 per million output tokens, which moves the short-prompt break-even to about 29%.

Start from the load you actually have. Traffic that only arrives during working hours, 40 of the week’s 168 hours, keeps a GPU busy at most 24% of the time. At October 8 prices, that rules out every DeepSeek row and leaves the 3090. Against the October 19 endpoint, the 3090 would need about 29%, so check current prices before you buy. A batch pipeline running around the clock can reach 70%, which brings both B200 rows into range against the median ZDR price, but neither against the cheapest.

In monthly terms, the rented 3090 costs $365 and pays for itself after about 170 million output tokens of our chat mix. The two-B200 node costs $5,402 a month at retail rent and has to serve 2.7 billion output tokens before it beats the cheapest ZDR endpoint.

The small model comes out ahead because its API prices are high compared with the hardware it needs. Providers charge $1.49 to $5.25 per million output tokens for Qwen3.8 27B, and a $0.50-an-hour card produces about 3.4 million of them an hour. DeepSeek V4.1 Flash is the opposite case. Twenty-four ZDR endpoints compete for its traffic, and the cheapest charges $0.18 per million output tokens.

The 3090’s low cost comes with a speed trade-off. It reaches that throughput by serving 64 requests at once, at about 15 tokens per second each. That’s slow for a chat interface, and slower still for a reasoning model that writes thousands of hidden tokens before its answer. Across the 18 API endpoints for Qwen3.8 27B, the median ran at 45 tokens per second in the 30 minutes before our pull, and the fastest at 474.

Why do published self-hosting costs look so low?

If you’ve seen self-hosting quoted at around a cent per million tokens, the token mix is usually the reason. InferenceX puts DeepSeek V4.1 Flash on H200 at $0.012 per million tokens at hyperscaler pricing. That figure is per million total tokens, and in its agentic workload, 99% of tokens are input, 95% of them served from the prefix cache. Through the API, cached input for that model costs $0.0007 to $0.20 per million, so those tokens are cheap either way.

Priced per output token, the same H200 at the $2.90 retail rent costs $3.49 per million before any input is counted. The cheapest ZDR endpoint charges $0.18.

Do you need to self-host for privacy?

Privacy is the other common reason to self-host, and you can get most of it through the API. Setting provider.zdr on a request limits it to endpoints whose providers don’t store your prompts or completions. If your ZDR setting also allows Safety Retention endpoints, those may keep prompts and responses to detect abuse, so choose ZDR only when you need no retention at all. Providers with implicit caching may also hold derived KV-cache data for a short time, as our implicit caching explainer describes. You can also enforce ZDR account-wide, per model group, or in a guardrail that covers specific API keys. On the Business and Enterprise plans, requests sent to eu.openrouter.ai or us.openrouter.ai run only on endpoints in that region, and they fail rather than leave it.

This script tries both, including two requests we expected to fail:

Output recorded October 8, 2026:

Both filters run before we pick a provider, so they shrink the list of candidate endpoints instead of rerouting after the fact. With zdr on, DeepSeek’s own API drops out because it retains data. Pinning the request to that provider with only leaves no endpoint at all, which returns the first 404. The EU domain keeps only endpoints we’ve onboarded in Europe. DeepSeek V4.1 Flash had none, so the third request failed instead of quietly going to the US. To use EU routing, pick a model with EU endpoints from GET /api/v1/models?region=eu, which listed 72 models, including GLM 5.3.

ZDR has one limit. Your prompts are still processed on a provider’s GPU, so it doesn’t give you control of the hardware. If your policy requires the model to run on a deployment you control, Private Endpoints (Enterprise plan) add your own dedicated or fine-tuned deployment to a catalog model’s provider list. It must be hosted with a provider we support for BYOK, your team calls it with the same slug, and public endpoints stay available as fallbacks if you list them. If data must never leave hardware you operate, self-host and size the hardware from the break-even table above. Our ZDR explainer and data residency guide compare the options in more detail.

Can you self-host and use the API together?

Yes, and for most teams the break-even table points to this setup. A GPU only pays off at high load, so size your own for the steady base of your traffic, where it stays busy, and send the peaks to the API, where you pay only for what you use.

This works because vLLM and SGLang expose the same OpenAI-compatible chat API we do. A client moves between them by changing the base URL, key and model name. Fall back to the API on a connection error, or on the 429 or 503 your server returns when its queue is full. Set provider.zdr on the API side so overflow traffic keeps zero data retention. If your policy requires prompts to stay on hardware you control, don’t overflow to the API.

Two things can go wrong with this setup. If your local copy runs at a different precision from the API endpoint, the same prompt can get different answers, so pin the API side to a matching quantization with provider routing. And a conversation that switches backends partway through loses its prompt cache, so keep each conversation on one side once it starts.

Frequently asked questions

Is self-hosting an LLM worth it?

For cost alone, only if you can keep the GPU busy. In our numbers, Qwen3.8 27B on a rented RTX 3090 breaks even at 6 to 12% load. DeepSeek V4.1 Flash on rented data-center GPUs needs 37% load or more, and against the cheapest zero data retention endpoint it often never breaks even. Privacy and control can justify self-hosting at any load.

Which LLM is best for self-hosting?

Models that fit on one GPU are the best fit, because they avoid multi-GPU servers and their API prices leave the most room to save. Qwen3.8 27B, for example, runs on one 24 GB card at 4 bits. Our rankings and the open-weight vs closed-weight comparison show which models developers use most.

What is the cheapest self-hosted LLM option?

A GPU you already own. An RTX 3090 limited to 250 W costs about $0.05 an hour in electricity at the July 2026 US residential average, so it beats the cheapest zero data retention endpoint for Qwen3.8 27B once it’s busy about 1% of the time. The trade-offs are slower responses for each user and your own time spent on maintenance.

Can I host my own model and still use it through OpenRouter?

Yes, on the Enterprise plan. Private Endpoints route a catalog model’s slug to your own dedicated or fine-tuned deployment, hosted with a provider we support for BYOK. Only the users and organizations you approve can see or call it.

Next steps

• Zero Data Retention: enforce ZDR per request, per model group, in a guardrail or account-wide.

Zero Data Retention: enforce ZDR per request, per model group, in a guardrail or account-wide.

• In-Region Routing: send requests through the EU and US domains, and enforce a region with a guardrail.

In-Region Routing: send requests through the EU and US domains, and enforce a region with a guardrail.

• The model pages for DeepSeek V4.1 Flash and Qwen3.8 27B list current per-provider prices and throughput.

The model pages for DeepSeek V4.1 Flash and Qwen3.8 27B list current per-provider prices and throughput.