> Source: https://souverana.ch/insights/llm-kosten-en/
> Language: en

# What LLMs really cost: the price per token is misleading

In one in three model comparisons, the model with the lower list price was the more expensive one in practice (study, May 2026). What you actually pay is not on the price list.

Analysis · Published 7 Jul 2026 · Updated 21 Oct 2026 · Joel Barmettler

## Why the price per token is misleading

**The model listed as cheaper is often not the cheaper one: in a May 2026 study by researchers including Stanford, the lower-listed model proved dearer in practice in one in three comparisons. Token consumption per task, the discount on repeated requests (prompt caching) and the quality at which the provider hosts the model decide the outcome.**

**In brief**

-   The list price says little about the bill: for the same task, one model sometimes consumes a thousand times as many tokens (small pieces of text) as another. Cheap tokens then buy you little.
-   Coding assistants and agentic applications, which work through many steps on their own, send the same context to the model hundreds of times. In a test month of our own via the DeepSeek API, the same usage cost 2.09 instead of 13.20 dollars with prompt caching (86 per cent of 30 million input tokens were cache hits).
-   The same model name does not mean the same model everywhere: many hosts, meaning providers that run models on their own servers and rent them out, use compressed variants without disclosing it.
-   Do not rely on price lists. Measure what your tasks actually cost with a given provider.

In advisory mandates we regularly compare models and hosts. The first question is almost always: “Which model is the cheapest?” The honest answer: that is not in the price list. Three factors decide what you end up paying, and all three are invisible at first glance.

This article opens our series on buying inference, meaning what language models really cost in operation. The two parts that follow examine the biggest cost lever, prompt caching.

**Series · Buying inference**

1.  What LLMs really cost: the price per token is misleading (this part)
2.  [Prompt caching: why agents cost more to run in Switzerland](https://souverana.ch/en/insights/prompt-caching/)
3.  [Prompt caching in operation: what a host must set up](https://souverana.ch/en/insights/prompt-caching-betrieb/)
4.  [How DeepSeek can be so cheap: a look at its operation](https://souverana.ch/en/insights/deepseek-inferenz-betrieb/)

## Factor 1: models consume very different numbers of tokens

Modern models “think” before they answer: they write internal intermediate steps that nobody reads but that are billed as normal. And they think for very different lengths of time. How different, a research team from Stanford, Berkeley, CMU and Microsoft Research measured in May 2026, across eight leading models, twelve task types and over 7 billion tokens.[1]

> **32 %** In one out of every three model comparisons in the study, the model listed as cheaper turned out to be the more expensive one in practice.
> 
> Source: Chen et al., The Price Reversal Phenomenon, arXiv:2603.23971, May 2026

The differences are enormous. For the same exam question, one model needed over 60,000 reasoning tokens, another got by with 25. For the same agentic task, one needed 7 steps, another 57. That is how Gemini 3 Flash, despite a token price 80 per cent lower than GPT-5.4, can end up costing more overall: 705 instead of 509 dollars for the same benchmark.[1]

The two charts show how much the picture shifts: on the left, the models ranked by list price; on the right, by what they actually cost in the benchmark.

**Chart: Ranked by list price**

*USD per 1M tokens (input + output)*

| Category | List price (USD/1M tokens) |
| --- | --- |
| Claude Opus 4.7 | 30 |
| GPT-5.4 | 17.5 |
| Gemini 3.1 Pro | 14 |
| Claude Haiku 4.5 | 6 |
| GPT-5.4 Mini | 5.25 |
| Kimi K2.6 | 4.95 |
| Gemini 3 Flash | 3.5 |
| MiniMax M2.7 | 1.5 |

**Chart: Ranked by measured cost**

*Total benchmark cost (USD)*

| Category | Measured cost (USD) |
| --- | --- |
| Claude Opus 4.7 | 708 |
| Gemini 3 Flash | 705 |
| Gemini 3.1 Pro | 677 |
| GPT-5.4 | 509 |
| Kimi K2.6 | 422 |
| Claude Haiku 4.5 | 421 |
| GPT-5.4 Mini | 325 |
| MiniMax M2.7 | 148 |

Source: Chen et al., arXiv:2603.23971, May 2026, Figure 1

Gemini 3 Flash, the second-cheapest model on the price list, sits at almost the top of the cost ranking: 705 dollars, level with the most expensive model on the list. The reason becomes clear from token consumption:

**Chart: List price vs. measured reasoning-token consumption (8 reasoning models, 12 task types)**

*List price (USD per 1M tokens, input + output) / Reasoning tokens in the benchmark (millions)*

| Category | List price (USD/1M tokens) | Reasoning tokens (M tokens) |
| --- | --- | --- |
| GPT-5.4 | 17.5 | 34.1 |
| GPT-5.4 Mini | 5.25 | 23.2 |
| Gemini 3.1 Pro | 14 | 91.9 |
| Gemini 3 Flash | 3.5 | 208.7 |
| Claude Opus 4.7 | 30 | 24.2 |
| Claude Haiku 4.5 | 6 | 37.8 |
| Kimi K2.6 | 4.95 | 91.6 |
| MiniMax M2.7 | 1.5 | 17.2 |

Source: Chen et al., arXiv:2603.23971, May 2026, Tables 3 and 5

The explanation lies in consumption: Gemini 3 Flash thinks six times longer in the benchmark than GPT-5.4. Claude Opus, the most expensive model on the list, is among the most frugal. Even the same model is not constant: between two runs with identical input, the difference in the study was up to a factor of 9.7.[1] Anyone comparing only price lists is comparing the wrong number.

## Factor 2: prompt caching decides agentic workloads

Prompt caching works like a short-term memory: the provider remembers what it has already processed. If your application sends the same context again, you pay almost nothing for it. At Mistral and Anthropic, a cache hit costs a tenth of the normal price; at DeepSeek, less than a hundredth: 0.0028 instead of 0.14 dollars per million tokens for the small model, 0.003625 instead of 0.435 dollars for the large one.

For chat applications, that is an advantage. For coding tools such as Claude Code or OpenCode, and for agentic applications, it is decisive, because these tools send nearly the same context hundreds of times per session.

We ran the numbers on our own billing. In June, an agentic setup was tested via the official DeepSeek API, 30 million input tokens in total. 86 per cent of them were cache hits. The monthly bill: 2.09 dollars. Without caching, the same month would have cost 13.20 dollars, same model, same provider. The first week of July showed the same picture, even more starkly, at a 94 per cent hit rate.

So before studying price lists, settle two questions: does the provider offer prompt caching, and how often does your application hit the cache? In Europe the choice is sobering: Mistral is the only EU provider we know of that offers prompt caching. [Anyone planning to self-host sovereignly](https://souverana.ch/en/insights/llm-selbst-hosten-kosten/) and running agentic applications should settle this point before signing.

## Factor 3: you do not know which model you are getting

The third factor is the trickiest, because you cannot see it. Open-weight models can be compressed, a technique called quantisation: the host stores the model’s weights at lower precision, say FP8 instead of FP16, and needs only half or a quarter of the memory for it. Running the model becomes markedly cheaper. Used moderately, nobody notices. Compressed too far, [answer quality drops measurably](https://souverana.ch/en/insights/llm-quantisierung/).

There is nothing wrong with that; quantisation is a legitimate lever. The problem is transparency. OVHcloud states its FP8 variants openly in its catalogue. Infomaniak discloses an FP8 suffix for some models and nothing for others. Kvant hosts the Swiss Apertus-70B as a 4-bit variant, while the majority of its catalogue carries no disclosure at all. So two hosts can offer the same model at the same price, and you still do not get the same model.

The model name on the price sheet is therefore no guarantee of quality. Ask at what precision a model is hosted. And test with your own tasks instead of public leaderboards, because those were run on the original model.

Want the model and host assessed independently, including cost on your own workload?

[AI architecture & solution selection](https://souverana.ch/en/leistungen/ki-architektur/)

## How to compare correctly

The good news: AI cost is controllable once you measure the right numbers. An afternoon of discipline is enough. Four steps lead there:

1.  **Define the workload**: Collect ten to twenty real tasks from your day-to-day work: the documents, the questions, the coding tasks your team actually has.
2.  **Measure end-to-end cost**: Run these tasks against the candidates and record total cost per completed task, not the price per token.
3.  **Ask the provider**: Get written confirmation: at what precision does the model run, is prompt caching available, and does the model version stay fixed?
4.  **Remeasure on every change**: New model, new host, new update: rerun the same benchmark. Otherwise nobody notices when the model listed as cheaper suddenly becomes the more expensive one.

## Sources

[1] Chen et al. «The Price Reversal Phenomenon». arXiv:2603.23971. May 2026. Primärquelle, Studie (Stanford, Berkeley, CMU, Microsoft Research).

The author

Joel Barmettler

AI Architect · Souverana, Zurich

Joel Barmettler guides Swiss companies from AI strategy to integration: sovereign, confidential and production-ready. He built the Swiss AI Hub as its architect and today co-owns its architecture; he personally leads every Souverana mandate. Mandates from one-person firms to Fortune 500 corporations.

[Book an intro call](https://meet.brevo.com/joel-barmettler/30-minute-meeting) [More about Souverana](https://souverana.ch/en/) [LinkedIn](https://www.linkedin.com/in/joel-barmettler-b9ab361b7)

## Frequently asked questions

Why is the cheapest LLM not automatically the cheapest to run?

Because models consume very different numbers of tokens for the same task. A model with cheap tokens often needs several times the reasoning tokens of another, meaning the internal steps before the answer, which are billed too. In a May 2026 Stanford study, the model listed as cheaper was the more expensive one in one out of every three comparisons.

What is prompt caching?

The provider stores prompt segments it has already processed. If the same context is sent again, those tokens cost, depending on the provider, 90 to over 99 percent less. For coding assistants and agentic applications, which resend the same context hundreds of times per session, this effect makes up most of the bill.

What does it mean when a model is quantised (FP8, 4-bit)?

The model's weights are stored at lower numerical precision. This markedly lowers the host's hardware requirements, but past a certain point it also lowers answer quality. A quantised model often carries the same name as the original; not every provider discloses whether, or how heavily, it has been quantised.

How do I compare LLM providers properly?

Most reliably with your own test: run the same tasks against both providers and measure what completing a task costs overall, caching included. Also ask at what precision the model is hosted, and how cache hits are billed.

## Glossary

-   **Token**: The smallest billing unit of a language model; a word is roughly one to three tokens. Providers charge per million tokens, separately for input and output.
-   **Agentic**: AI applications that work in multiple steps, on their own: they plan, call tools and repeat steps until the task is done. In doing so they send the same context many times over.
-   **API**: A programming interface through which software addresses a language model directly, without a chat interface. Usually billed per token.
-   **Prompt caching**: The provider stores prompt segments it has already processed. If the application sends the same context again, those tokens cost only a fraction of the normal price.
-   **Cache hit**: A hit means the start of a request is already in the cache and does not have to be computed again. The hit rate is the share of the input that comes from the cache.
-   **Quantisation**: Compressing a model by lowering the numerical precision of its weights (e.g. FP8 or 4-bit). Operating it becomes markedly cheaper; past a certain point, answer quality drops measurably.
-   **Inference**: Running a trained model in production: a request goes in, an answer comes out. Inference is what you buy from a host per token.
-   **Reasoning tokens**: Reasoning models generate internal intermediate steps ("thinking") before the actual answer. These reasoning tokens are invisible but are billed as output.
-   **Benchmark**: A standardised test with fixed tasks used to compare models. A good benchmark result says little by itself about your own task.
-   **Open weight**: Models whose weights are publicly available and may be self-hosted (e.g. Apertus, DeepSeek, Llama). A precondition for sovereign operation without vendor lock-in.
-   **Precision (BF16, FP8)**: How many bits a model uses to store its numbers. Fewer bits save memory and compute but can lower quality.