Self-hosting an LLM: GPT-5 performance for CHF 8’500

What can a self-hosted LLM actually do?

Enough for the daily work of twenty to thirty employees. A workstation costing around CHF 8’500 runs an open model at GPT-5.1 level, carrying correspondence, document search and translation. Demanding coding work and long agentic tasks stay out of scope. Models that no longer fit entirely into graphics memory run only overnight.

In brief

• Start with the tasks, only then with the hardware. What is meant to run in-house and how many people work with it at the same time determines everything that follows.
• For correspondence, document search and translation, a machine under the desk is enough. For demanding coding work, it is not.
• Think in concurrent requests, not headcount. Twenty to thirty employees do not push such a machine to its limit.
• Graphics memory decides which model fits. Speed is decided by the active parameters and the memory bandwidth: a 35-billion MoE model was twice as fast for us as a 27-billion model.
• If your application can wait, an entire further model class opens up. Overnight processing is the cheapest route to higher quality.
• Do not expect savings. Anyone who sets up their own hardware does it for control over the data.

In advisory mandates, law firms, medical practices and trust offices now raise this question regularly. Their data is especially sensitive, often subject to professional secrecy?Professional and official secrecyThe criminally protected duty of confidentiality of authorities (official secrecy, Art. 320 of the Swiss Criminal Code) and a list of professions named in the law (Art. 321), such as lawyers, doctors and pharmacists. It applies independently of data protection law and reaches further. Banking secrecy sits separately in the Banking Act.. A contract with a provider resolves that legally↗Law & Regulation · Part 3Professional secrecy and AI: what may go to the cloud'That does not work for us' is the standard answer to AI in law firms and medical practices. Art. 321 of the Swiss Criminal Code says something narrower, and most of daily work falls outside it entirely.Read article, yet for some records management simply wants the files never to leave the building.

To “which model should we take”, we answer with two questions of our own: Which tasks should run in-house? And how many people work with it at the same time? The model class follows from the answers, and only from that follows the machine.

We walked this path ourselves and measured for three weeks. What follows are our own figures from August 2026.

This article opens our five-part series on local inference, meaning the operation of language models on your own hardware. It tells what we built and measured; the technical parts that follow show how to size it yourself.

Tasks first, hardware second

What runs reliably on your own hardware are tasks with a clear frame and a checkable result: extracting information from documents, classifying incoming items, summarising, translating, anonymising, answering questions over your own case files. For a law firm or a practice, that is most of what AI is meant to do day to day.

Language is one of its strengths. The model we run solved 82 percent of a professionally translated German knowledge test. A second, machine-translated test of the same kind produced 88 percent; the six-point difference comes from the translation, not the model. German is not a bottleneck.

What remains difficult is long agentic?AgenticAI applications that work in multiple steps, on their own: they plan, call tools and repeat steps until the task is done. In doing so they send the same context many times over. work, a task spanning twenty tool calls↗Sovereign AI · Part 5Sovereign AI integration: when AI starts to actThe moment an agent acts instead of merely reading, a mistake becomes an incident. Integration is where your IP is built and where you have to keep control. How that works.Read article in a row, where a single formatting error breaks the chain. The same goes for the hardest reasoning tasks. Anyone who needs a top-tier coding assistant continues to buy it in.

So start by writing down ten to twenty real cases from your day to day. If most fall into the first list, reading on is worthwhile. If they fall into the second, save yourself the hardware.

Twenty employees are not twenty requests

The second question is almost always framed wrong. “Is this enough for 200 people” cannot be answered, because a machine does not carry a headcount, it carries concurrent requests.

Someone who reads an answer, checks it and types the next question occupies the machine for perhaps ten seconds per minute. If forty of two hundred people are active around lunchtime and each asks a question every two minutes, on average fewer than twenty requests run in parallel. For twenty to thirty employees, the number sits in the single digits.

Work out your own number before reading on. The graphics cards later depend on it, and we return to it when we get to the measurements.

The machine that follows from it

A language model has to fit entirely into the memory of the graphics card, or it becomes sluggish. Graphics memory is therefore the quantity everything revolves around when self-hosting, and the price per gigabyte varies widely. Professional cards offer 96 GB in a single slot and cost several times as much per gigabyte. An RTX 4090 from the gaming segment offers 24 GB for around CHF 1’700.

So, two of them. The rest of the list follows from that: a motherboard with two long slots, a power supply that can carry both cards, and plenty of system memory, whose purpose becomes clear further down.

Component	Quantity	CHF
ASUS TUF Gaming RTX 4090, 24 GB	2	approx. 1’700 each
AMD Ryzen 9 7950X	1	489
ASUS ROG Crosshair X670E Hero	1	475
Kingston FURY Beast DDR5-6000, 2 x 32 GB	2	909 each
Crucial T700, 4 TB	2	520 each
Corsair HX1500i, 1500 W	1	719
Case, cooling, small parts		approx. 460
Total		approx. 8’450

The whole thing is a desktop machine with two graphics cards, orderable from any Swiss retailer. It stands under a desk and plugs into a normal wall socket. What comes out of it is 48 GB of graphics memory and 128 GB of system memory.

That settles the hardware. What is missing is the model.

The search for a model starts with a leaderboard

48 GB stands ready. Hugging Face hosts tens of thousands of open-weight models?Open weightModels whose weights are publicly available and may be self-hosted (e.g. Apertus, DeepSeek, Llama). A precondition for sovereign operation without vendor lock-in., many of them variants of variants. We started with Artificial Analysis. The site runs models through the same set of tests and rolls the result up into an Intelligence Index, a number on a scale to 100. For our case, it has its own view for small open models, exactly the field that fits a machine like ours.

Before a list like that is useful, you need to be able to read the names. They look confusing and contain exactly the two numbers that matter.

Model name, decoded

Qwen 3.6 -35B -A3B

Qwen

The provider. Alibaba, in this case. It says nothing about capability, but a lot about maintenance, licence and how many sizes exist within the same family.

3.6

The generation. A higher number beats a lower one from the same family almost always, even at the same size.

-35B

Total parameters. 35 billion. This number determines how much graphics memory you need, because that much has to be loaded.

-A3B

Active parameters. Three billion. This number determines the speed, because only that much is computed per word. If the figure is missing, the model computes with everything it has.

The two red numbers do different things. The first costs you memory, the second buys you speed.

If the second number block is missing, it is a dense model: it computes with every single parameter for every single word. If an A appears there, it is an MoE model?MoEMixture of Experts: a model that stores many sub-models ("experts") but computes only a few of them per token. All parameters are stored; only a fraction are used for any given calculation., split into many experts with different specialities, of which only a handful are engaged per word. This design is the reason a larger model can answer faster than a smaller one.

Three classes, and why only one is in play

Sorted by this pattern, the open models fall into three classes as of August 2026.

Small, up to around four billion parameters. These models run on a laptop, the smallest on a phone. They summarise texts, sort incoming items and locate passages. For anything that has to get several steps right in sequence, they are too weak.

Medium, roughly nine to forty billion parameters. Qwen, Gemma, Mistral and Apertus sit in this class. Quantised, a model of this size fits into 24 to 48 GB of graphics memory.

The open flagship, 300 billion to 2.8 trillion parameters. DeepSeek, Kimi, GLM. These models compete against the closed flagship models and come close. For that, they need dozens of data-centre cards and acquisition costs running into the millions.

In the index, the gap between these worlds looks like this:

Model	Index
Claude Opus 5 (proprietary, flagship)	63.1
Kimi K3, 2.8 trillion parameters (open flagship)	59.7
Qwen3.6-27B (fits entirely in 48 GB)	37.7
GPT-5.1, for comparison	37.5
GPT-4o (March 2025)	12.3

Source: Artificial Analysis, retrieved 12 August 2026.

The last three rows are the good news. The model that fits on a machine under the desk is level with GPT-5.1 and far above GPT-4o, the model most people first came to know ChatGPT by.

From the medium class, we took Qwen, for three reasons. At the time of measurement, it was the highest-ranked open model that fits entirely into 48 GB. The same family exists in many sizes and as both a dense and an MoE variant, which is what makes comparisons possible in the first place. And ready-made 4-bit versions exist from several independent providers.

A leaderboard has its limits, though. Anyone who knows an exam can train for it, and an index says nothing about your documents, your language and your tools. So we downloaded the candidates and remeasured them ourselves.

What size buys you

We measured with tests whose result can be counted: arithmetic problems in text form, small coding tasks with an executable solution, and German knowledge questions in multiple-choice format. The figure given is the share of correct answers in each case. The first four rows come from the same model family and ran without quantisation?QuantisationCompressing a model by lowering the numerical precision of its weights (e.g. FP8 or 4-bit). Operating it becomes markedly cheaper; past a certain point, answer quality drops measurably.. The fifth is the model we actually run on this machine:

Size	Arithmetic	Coding	German knowledge
0.8B	38 %	18 %	41 %
2B	57 %	22 %	55 %
4B	73 %	39 %	66 %
9B	87 %	55 %	74 %
27B (4-bit, newer generation)	66 %	74 %	82 %

Below nine billion parameters, a model is not fit for professional work. Four billion does not solve even two out of five coding tasks and does not know a third of the German knowledge questions. For summarising and sorting, that is enough. As soon as someone relies on the result, it becomes too little.

It becomes usable in the class around thirty billion. The 27-billion model solves three out of four coding tasks, where the 9-billion model manages just half. This is exactly the class that fits on 48 GB of graphics memory. On arithmetic, it stands out at 66 against 87 percent. Three reasons account for that, and none of them is reassuring: the model belongs to a different generation than the rung above it, it ran quantised to 4 bit rather than unquantised, and it thinks before answering, for which we gave it no extra token budget. This row deserves a remeasurement.

For the hardest tasks, you need more than a hundred billion parameters. And with that we arrive at the actual point, because by then the hardware has long since been bought.

What determines speed: active parameters and bandwidth

From here, the question changes. The 48 GB decide which model fits at all. How fast it then answers is decided by two other quantities: how many parameters actually compute per word, and how fast the card reads its memory.

The first quantity is shown by two models on the same machine, under the same load, with eight concurrent users. Only the model changes.

Configuration	Tokens per second and person	Wait time to first character
Qwen3.6-35B (MoE), entirely on the cards	104.6	1.6 s
Qwen3.6-27B, entirely on the cards	51.6	5.8 s

The larger model is twice as fast and keeps its users waiting a quarter as long. Qwen3.6-35B computes with three of its 35 billion parameters per word, the 27-billion model with all of them. For memory requirements, the total count matters; for speed, the active one does. This is exactly why we recommend MoE models in this class: they occupy the memory of a large model and compute at the speed of a small one.

The second quantity is memory bandwidth, and it explains a surprise in hardware purchasing. For every single word, the active weights have to pass through the card’s memory once. An RTX 4090 reads around 1,000 GB per second for that. Nvidia’s L4, a widely used data-centre card with the same amount of memory, reaches 300 GB per second (manufacturer figures). For inference, that means: the more expensive server card answers more slowly than the gaming card. It is designed for 72 watts, half-height and continuous operation, not for bandwidth.

So as long as the model fits entirely onto the cards, there is no trade-off between quality and speed. That trade-off only begins once it no longer fits, and that is exactly what we measure further down with the 284-billion model.

Now to your number from the second question. Besides the model, every running conversation also takes up space on the card, because the model has to carry its history along. This buffer is called the KV cache?KV cacheThe working memory in which a model holds the conversation so far. It grows with the text length and with every concurrent user, and often takes up more graphics memory than the model itself., and it grows with the text length and with every concurrent user.

It explains why two cards deliver more than one does. On a single RTX 4090, a 27-billion model takes up so much space that only around 15,000 tokens remain for all conversations combined, not even a single longer request. On two cards, it is 337,000. The second card does not make the model smarter, it makes it operable.

Measured under realistic load, meaning 10,000 input tokens and 2,000 output tokens per request:

The middle column counts all users together, the right-hand one measures how fast the text appears for a single person. The two cannot be converted into each other, because a request spends part of its time reading rather than writing.

Concurrent requests	Tokens per second, total	Per person	Longest wait time
1	65.8	74.3	3.5 s
4	169.0	59.7	13.7 s
8	240.6	51.4	27.5 s
16	301.6	39.3	55.1 s

Total throughput keeps rising to the end, and the per-person writing speed stays readable. What tips over is the longest wait time: from 3.5 to 55 seconds. At 10,000 input tokens, the card spends most of its time reading the requests, and these reading phases get in each other’s way.

Eight concurrent requests, then, this machine handles comfortably. At sixteen, no response time can be promised any more. If your number from the second question sits below that, the machine fits.

The extreme case: 284 billion parameters

How far this can be pushed is shown by the most extreme case. DeepSeek V4 Flash has 284 billion parameters and 51.8 index points, closing a good two thirds of the gap to the open flagship. Does a model like that run on a machine costing CHF 8’500?

It runs. In 10.9 GB of graphics memory.

What makes that possible is the design pushed to its most extreme form. The full name reads DeepSeek-V4-Flash-284B-A13B: 284 billion parameters are stored, 13 billion are computed per word. At a ratio like that, the rarely used experts are allowed to sit in cheap system memory, and only the ones actually needed travel to the graphics card. That is exactly what the 128 GB of system memory in the parts list is for.

The price paid for that is time. We asked a single German question and timed it: after 10.7 seconds the model began working, the first character became visible after 37.3 seconds, and the three-sentence answer stood after 53.9 seconds. In between lay 196 reasoning tokens that nobody gets to see.

The bar counts down, and that is exactly the experience: a good half minute in front of an empty screen. Here the trade-off shows up that did not exist for the models sitting on the cards. The same machine that delivered 104.6 tokens per second and person with the 35-billion model now writes 7.9, and the wait rises from 1.6 to 37.3 seconds. The reason is not the parameter count as such: the experts sit in system memory, and that is connected at a fraction of the speed of the graphics card’s memory.

What this means for your company

For a business with twenty to thirty employees that writes correspondence, searches documents and translates, a machine of your own is feasible. It costs around CHF 8’500, stands under a desk and delivers a model at GPT-5.1 level. Anyone who also wants to do demanding coding or run agents over long chains continues to buy that part in.

If a task needs more quality and can wait, overnight processing opens up, all the way up to models with 284 billion parameters. A year ago, a model of this class only ran in a data centre.

For model choice, our recommendation is: MoE over dense, because the ratio of quality to speed comes out best there. Today that means Qwen. In a few months it will mean something else, and this paragraph will be out of date. So check the leaderboard yourself and remeasure the candidate on your own tasks before committing.

Two things to close, neither of which appears in any measurement series. First, the arithmetic: over three years, the machine has to run at full load around 39 percent of the time to come out cheaper than Swiss provider prices; over two years, 57 percent. How these figures come about, we work through in the final part of this series. Second, a question on which entire projects in our mandates regularly hinge: Who operates the machine when it stops answering on a Friday evening?

Anyone who has answered both can decide. Going by our figures, the reason for a yes lies in control over the data, almost never in the price.

1. 01

   Capture the tasks

   Write down ten to twenty real cases from your day to day, with the typical document length. Without this list, neither the model nor the hardware can be chosen sensibly.

2. 02

   Estimate the concurrent requests

   Work it out from the number of people active at the same time and their thinking pauses. The graphics cards depend on this number.

3. 03

   Measure with rented hardware

   A GPU hour with a Swiss or European provider costs a few francs. Only once your own tasks run cleanly there does the purchase pay off.

4. 04

   Staff the operation

   Updates, model swaps, monitoring and restarts are recurring work. Clarify who takes it on before the cards are ordered.

The remaining parts of the series go deeper into what only appeared in outline here: the memory arithmetic, quantisation, engine choice and finally the cost question.

Considering running a language model in-house? The first question is usually whether the quality is even good enough.

The headlines suggest it is: open models now trail the proprietary flagship by only 3.4 index points (Artificial Analysis, 12 August 2026). That figure, however, applies to models with trillions of parameters, ones nobody puts under a desk.

So we measured what actually runs on a machine costing CHF 8’450 with two RTX 4090s. The best model that fits entirely into the 48 GB is Qwen3.6-27B at 37.7 points, level with GPT-5.1. 22 points separate it from the open flagship. One special case remains: DeepSeek V4 Flash, a mixture-of-experts model with its experts held in system memory, reaches 51.8 points at 7.9 tokens per second, overnight processing rather than dialogue.

Our conclusion: for correspondence, document search and translation in-house, that is enough today. Demanding coding work and long agentic chains, you continue to buy in for now.

The article carries the parts list, all the measurements, and the explanation for why the larger of the two models is the faster one. Link in the comments.

#AI #DataSovereignty #LLM #Switzerland