LLM hosting for business: managed API vs self-hosting vs on-prem, with real monthly costs

What are the three ways to host an LLM?

Every option runs the same kind of model on the same kind of chip. What changes is who owns the hardware, who keeps it running and how the bill is shaped.

What does a managed LLM API cost per month?

API pricing is per token, split between what you send (input) and what the model writes back (output). To turn that into a monthly figure you need the size of a typical request. We use the same yardstick as our Token Factory page: a customer support answer of about 500 input tokens (instructions plus the question) and 300 output tokens.

The table uses current Token Factory prices for a small, a mid-size and a large open-source model, in euros with VAT included, for three volumes of answers a day over a 30-day month.

Two things stand out. For the same answers, the large model costs many times what the small one does, so test the smaller models first and only move up when quality demands it. And the cost scales down to zero: a quiet month costs almost nothing, which a rented GPU cannot do.

What does self-hosting on a rented GPU cost per month?

Self-hosting turns a variable cost into a fixed one. You rent the GPU, deploy the model with an open-source engine such as vLLM or SGLang, and pay for every hour it exists, busy or idle.

These are our dedicated GPU prices: a physical MIG slice of an NVIDIA B200 or the whole card, by the hour or reserved by the month. A full month of hourly use is shown at 730 hours, next to the monthly reservation.

For reference, two other providers list B200 capacity by the hour. Their prices are in US dollars, as published, and are not converted.

Together AI (GPU clusters, on demand, per GPU) and Modal (published per second, multiplied by 3,600 here) checked on 9 November 2026. Check each page for minimum sizes and regions. Sources: www.together.ai/pricing, modal.com/pricing

On top of the GPU you pay for the disk that holds the model weights and your data, and for the time of whoever deploys, monitors and updates the serving stack. That last line is usually the largest one and never appears on a price list.

How much GPU does each model need?

The size of the GPU you need is set by the model, not by your traffic. This is the hardware each model in our catalogue is deployed on for Token Factory, which gives a realistic floor for self-hosting the same model.

Small and mid-size models fit on a slice of one card. A 40-billion-parameter dense model takes a whole B200, and the largest mixture-of-experts model in the catalogue needs eight. Longer contexts and more concurrent users need more memory on top, so treat these as the starting point.

When does a rented GPU beat the API?

Take Qwen3.8 27B, which costs €0.14 per 1,000 answers through the API. A reserved 23 GB MIG slice of a B200 costs €723 a month. You need around 5,100,000 answers a month, about 172,000 a day, before the GPU is cheaper, and only if that one slice can actually serve them at the speed your users expect.

For a large model the gap is wider. GLM 5.3 NVFP4 is deployed on 8 whole B200s; reserving that many costs €34,164 a month, which matches its API bill only at about 654,000 answers a day, every day.

How many requests a GPU can serve depends on the model, the prompt length, the output length and how many users arrive at once. We do not publish a throughput figure for you to plug in here: measure it with your own prompts, with a serving benchmark such as the one included in vLLM, before you commit to a reservation.

There are good reasons to self-host that are not about price: a fine-tuned model of your own, a model nobody offers as an API, or a requirement to have the hardware reserved for you alone. If one of those applies, the GPU is the right answer even below the break-even point.

What does an on-prem LLM really cost?

The reference system for an on-prem B200 deployment is NVIDIA's DGX B200: 8 GPUs, 1,440 GB of GPU memory in total, 10 rack units and a maximum power draw of about 14.3 kW, according to NVIDIA's own product page. That page lists no price; it sends you to sales. We have not found a primary source with a price we could quote, so this section lists what goes into the cost instead of guessing a number.

One line you can calculate yourself is electricity. At maximum draw, 14.3 kW for 730 hours is about 10,439 kWh a month for the system alone, before cooling. Multiply by your own tariff.

1. 1The servers themselves, quoted by NVIDIA or a reseller, plus spare parts and support contracts.
2. 2Power and cooling sized for the full draw, and rack space in a data centre that can deliver it.
3. 3High-speed networking between servers if the model spans more than one machine.
4. 4People: someone has to install drivers, keep the serving stack updated and answer the pager at night.
5. 5Depreciation: the hardware loses value as each new GPU generation arrives, whether you use it or not.

NVIDIA DGX B200 specifications checked on 9 November 2026. Sources: www.nvidia.com/en-us/data-center/dgx-b200/

How this works on GPU Flow

Token Factory is our managed API: open-source language, vision and audio models behind an OpenAI-compatible endpoint, so existing code only needs a new base URL and key. You load credit in euros and it goes down as you use it; when it runs out the API answers with a 402 error until you top up, and nothing is charged to your card automatically. Limits start at 1,000 requests per minute per key, raisable to 5,000, with 5,000 per account. What you send is never used to train any model.

When the API is not enough, a dedicated GPU gives you a hardware-isolated MIG slice of a B200, or the whole card, in a data centre in Spain. You get SSH access with drivers ready and run your own software on it, by the hour or reserved by the month at 10% below the hourly rate. Both are invoiced in euros, VAT included.

A common pattern is to start on the API, measure real traffic for a few weeks, and only then decide whether any part of it justifies its own GPU.

Which option should your business choose?

1. 1Traffic below the break-even point above, or traffic that varies a lot from day to day: use a managed API.
2. 2You need a model nobody serves, a fine-tuned model of your own, or hardware reserved only for you: rent a dedicated GPU.
3. 3Steady, very high volume that you have measured, on one model that fits a single card: compare a reserved GPU against your actual API bill.
4. 4A legal or contractual need to own the physical machines, and a team that already runs a data centre: consider on-prem.

Related

• The cheapest LLM APIs in 2026: price per million tokens compared (OpenAI, Together, OpenRouter, GPU Flow)Price per million tokens on OpenAI, Together AI, OpenRouter and GPU Flow, each in its currency, with the cost of 1,000 requests and what makes a model cheap.
• GPU Flow vs Modal: GPUs, sandboxes and inferenceWhat a B200, a sandbox for an agent and inference cost on each, with Modal’s prices checked on its own site.