> Quelle: https://souverana.ch/en/insights/llm-kosten/
> Sprache: en

# What LLMs really cost: the price per token is misleading

Comparing price lists is time wasted: token consumption, cache hit rate and quantisation decide what you actually pay.

Analysis · Published 7 Jul 2026 · Updated 24 Aug 2026 · Joel Barmettler

## Why the price per token is misleading

**The price per token says little about what a language model actually costs to run. What matters is how many tokens a model consumes for a task, whether the provider bills repeated requests more cheaply (prompt caching), and at what quality it hosts the model. That is why the cheapest model on the price list often ends up one of the most expensive.**

**In brief**

-   For the same task, one model sometimes consumes a thousand times more tokens? than another. Cheap tokens then buy you little.
-   Coding assistants and agentic? applications send the same context hundreds of times. With prompt caching these repeats cost almost nothing; without it, you pay the full price every time.
-   The same model name does not mean the same model everywhere: many hosts run compressed variants? without disclosing it.
-   So do not rely on price lists. Measure what your tasks actually cost with a given provider.

In advisory mandates we regularly compare models and hosts. The first question is almost always: “Which model is the cheapest?” The honest answer: that is not in the price list. Three factors decide what you end up paying, and all three are invisible at first glance.

## Factor 1: models consume very different numbers of tokens

Modern models “think” before they answer: they write internal intermediate steps that nobody reads but that are billed as normal. And they think for very different lengths of time. How different, a research team from Stanford, Berkeley, CMU and Microsoft Research measured in May 2026, across eight leading models, twelve task types and over 7 billion tokens.

Key figure

32 %

In one out of every three model comparisons in the study, the model listed as cheaper turned out to be the more expensive one in practice.

Source: Chen et al., The Price Reversal Phenomenon, arXiv:2603.23971, May 2026

The differences are enormous. For the same exam question, one model needed over 60,000 reasoning tokens?, another got by with 25. For the same agentic task, one needed 7 steps, another 57. That is how a model with an 80 percent lower token price can end up costing more overall: 705 instead of 509 dollars for the same benchmark.

The two charts show how much the picture shifts: on the left, the models ranked by list price; on the right, by what they actually cost in the benchmark.

Source: Chen et al., arXiv:2603.23971, May 2026, Figure 1

Gemini 3 Flash, the second-cheapest model on the price list, sits at practically the top of the cost ranking: 705 dollars, level with the most expensive model on the list. Why becomes clear once you look at token consumption:

The explanation lies in consumption: Gemini 3 Flash thinks six times longer in the benchmark than GPT-5.4. Claude Opus, the most expensive model on the list, is among the most frugal. And even the same model is not constant: between two runs with identical input, the study found a gap of up to a factor of 9.7. Anyone comparing only price lists is comparing the wrong number.

## Factor 2: prompt caching decides agentic workloads

Prompt caching works like a short-term memory: the provider remembers what it has already processed. If your application sends the same context again, it pays almost nothing for it. At Mistral and Anthropic, a cache hit costs a tenth of the normal price; at DeepSeek, less than a hundredth: 0.0028 instead of 0.14 dollars per million tokens for the small model, 0.003625 instead of 0.435 dollars for the large one.

For chat applications, that is a nice bonus. For coding tools such as Claude Code or OpenCode, and for agentic applications, it is decisive, because these tools send nearly the same context hundreds of times per session.

We ran the numbers on our own billing. In June, an agentic setup ran on trial over the official DeepSeek API, 30 million input tokens in total. 86 percent of them were cache hits. The monthly bill: 2.09 dollars. Without caching, the same month would have cost 13.20 dollars, same model, same provider. The first week of July showed the same picture, even more starkly, at a 94 percent hit rate.

So before studying price lists, settle two questions: does the provider offer prompt caching, and how often does your application hit the cache? In Europe the choice is sobering: Mistral is the only EU provider we know of that offers prompt caching. [Anyone planning to self-host sovereignly↗](/en/insights/llm-selbst-hosten-kosten/) and running agentic applications should settle this point before signing.

## Factor 3: you do not know which model you are getting

The third factor is the trickiest, because you cannot see it. Open-weight models? can be compressed, a technique called quantisation: the host stores the model’s weights at lower precision, say FP8 instead of FP16, and needs only half or a quarter of the memory for it. Running the model becomes markedly cheaper. Used moderately, nobody notices. Compressed too far, [answer quality drops measurably↗](/en/insights/llm-quantisierung/).

There is nothing wrong with that; quantisation is a legitimate lever. The problem is transparency. OVHcloud states its FP8 variants openly in its catalogue. Infomaniak discloses an FP8 suffix for some models and nothing for others. Kvant hosts the Swiss Apertus-70B as a 4-bit variant, while the majority of its catalogue carries no disclosure at all. So two hosts can offer the same model at the same price, and you still do not get the same model.

The model name on the price sheet is therefore no guarantee of quality. Ask at what precision a model is hosted. And test with your own tasks rather than public leaderboards, because those were run against the original.

Want the model and host assessed independently, including cost on your own workload?

[AI architecture & solution selection](/en/leistungen/ki-architektur/)

## How to compare correctly

The good news: AI cost is controllable once you measure the right numbers. Getting there is not a research project, just an afternoon of discipline.

1.  01
    
    Define the workload
    
    Capture ten to twenty real tasks from your day to day: the documents, the questions, the coding tasks your team actually has.
    
2.  02
    
    Measure end-to-end cost
    
    Run these tasks against the candidates and record total cost per completed task, not the price per token.
    
3.  03
    
    Question the provider
    
    Get written confirmation: at what precision does the model run, is prompt caching available, and does the model version stay fixed?
    
4.  04
    
    Remeasure on every change
    
    New model, new host, new update: rerun the same benchmark. Price reversals happen quietly.
    

The author

![Portrait of Joel Barmettler](/_astro/joel-barmettler.CGKHGWrV_sJ0IG.webp)

Joel Barmettler

AI Architect · Souverana, Zurich

Joel Barmettler guides Swiss companies from AI strategy to integration: sovereign, confidential and production-ready. He built the Swiss AI Hub as its architect and today co-owns its architecture; he personally leads every Souverana mandate. Mandates from one-person firms to Fortune 500 corporations.

[Book an intro call](https://meet.brevo.com/joel-barmettler/30-minute-meeting) [More about Souverana](/en/) [LinkedIn](https://www.linkedin.com/in/joel-barmettler-b9ab361b7)

## Frequently asked questions

Why is the cheapest LLM not automatically the cheapest to run?

Because models consume very different numbers of tokens for the same task. A model with cheap tokens often needs a multiple of the reasoning steps. In a Stanford study from May 2026, the model listed as cheaper turned out to be the more expensive one, all in, in one out of every three comparisons.

What is prompt caching?

The provider stores prompt segments it has already processed. If the same context is sent again, those tokens cost, depending on the provider, 90 to over 99 percent less. For coding assistants and agentic applications, which resend the same context hundreds of times per session, this effect dominates the bill.

What does it mean when a model is quantised (FP8, 4-bit)?

The model's weights are stored at lower numerical precision. This markedly lowers the host's hardware requirements, but past a certain point it also lowers answer quality. A quantised model often carries the same name as the original; not every provider discloses whether, or how heavily, it has been quantised.

How do I compare LLM providers properly?

Most reliably with your own test: run the same tasks against both providers and measure what completing a task costs overall, caching included. Also ask at what precision the model is hosted, and how cache hits are billed.

LinkedIn

## Share this article

Ready-formatted graphics and a suggested post for your LinkedIn feed: download, copy, post.

![Slopegraph of eight language models: on the left, ranked by list price per million tokens; on the right, by measured cost for the same benchmark. Gemini 3 Flash rises from rank seven to rank two.](/media/llm-kosten-en/infografik.png)

[Download infographic (PNG)](/media/llm-kosten-en/infografik.png)

Suggested post

Do you compare AI providers by the price per million tokens? Then you are comparing the number that says least about your actual bill.

For Swiss companies it gets worse: sovereign hosting starts from a weaker position here. Mistral is the only EU provider we know of with prompt caching, and at Infomaniak as well as kvant, only some models disclose which quantisation they run.

What the price sheet hides, a research team from Stanford, Berkeley, CMU and Microsoft Research measured in May 2026: eight leading models, twelve task types, over seven billion tokens. In one out of every three model comparisons, the model listed as cheaper turned out to be the more expensive one in practice. Gemini 3 Flash, for instance, has the second-cheapest list price and the second-highest bill, because it consumes 208.7 million reasoning tokens against 24.2 million for Claude Opus.

The second lever is caching: in our own test month over the DeepSeek API, 86 percent of 30 million input tokens were cache hits. The bill: 2.09 dollars instead of 13.20, same model, same provider.

Our conclusion: price lists are not fit for comparison. Measure ten to twenty real tasks end to end, and remeasure with every change.

The article carries the benchmark data, the four steps for your own comparison, ten questions for your provider and the EU and Swiss provider table. Link in the comments.

#AICosts #LLM #Switzerland #DataSovereignty