Apertus: the Swiss language model and how it was built

What is Apertus?

With the Swiss language model Apertus (ETH Zurich, EPFL, CSCS), you can check what it was trained on. Of five open models compared, only OLMo 3 allows the same. The price: Apertus trails OLMo 3 on mathematics and code, and leaving out data of unclear licence cost 5.8 per cent in performance in the trial.

The key points in brief

• Read leaderboards with caution. The team deliberately left out data with an unclear or non-commercial licence and measured what that costs: 5.8 per cent of performance on average (experiment with Apertus 8B in fine-tuning?Fine-tuningFurther training of a pretrained model with selected examples so that it follows instructions or masters a specialised task. Also called post-training., Tech Report 1.0).
• If you need to prove what a model was trained on, that is possible with Apertus and OLMo 3: training data, pipeline and intermediate checkpoints are public. Llama 4, Qwen3 and Gemma 3 disclose only the weights (model cards, as of October 2026).
• Test specialist applications on your own cases. In a study by the team with models of 1.5 billion parameters?ParametersThe learned numerical values of a language model. Their number describes its size: "8B" means eight billion parameters. Larger models can do more but need more GPU memory., the opt-outs cost almost nothing in general knowledge, but measurable performance on biomedical tasks.
• If you run Apertus yourself, you need your own protection for personal data?Personal dataAny information relating to an identified or identifiable person, so customer records, HR files or even an email address. As soon as such data is involved, the revFADP applies. in the answers. Before training, only email addresses, IP addresses and IBAN numbers are replaced, and according to the model card the model has no filter for its outputs.
• Ask every model provider for the summary of training data that the EU AI Act?EU AI ActThe European Union's AI regulation, in force since August 2024 and applicable in stages. It is a product-safety law: it sorts AI applications by risk and is aimed above all at those who bring AI to the EU market. requires. In March 2026, the summary for Apertus received the top grade in a review by researchers at Trinity College Dublin and the Mozilla Foundation. None was available from OpenAI, Google or xAI at the time.

Two days after the release in September 2025, the blog clickomania ran the headline “So wird das nichts mit dem Schweizer ChatGPT-Killer” (roughly: this will not work out as a Swiss ChatGPT killer).[1] The NZZ wrote in March 2026: “Nett, aber nicht praxistauglich” (nice, but not fit for practice).[2] In October 2026, SRF asked why Apertus lags behind the commercial competition.[3] All three measure a research project against finished products.

This article reads Apertus differently: as Switzerland’s ability to train large language models itself, and as a series of documented decisions about which data a model may lawfully learn from. It opens our three-part series “Understanding Apertus”.

The model family from 1.0 to 2.0

Version	Released	Sizes	What is new
1.0	2 September 2025	8B, 70B	15 trillion tokens?TokenThe smallest billing unit of a language model; a word is roughly one to three tokens. Providers charge per million tokens, separately for input and output. of training, context 65,536 tokens[4]
Mini (1.1)	May and June 2026	0.5B, 1.5B, 4B	distilled from the 8B, 16 models including quantisations?QuantisationCompressing a model by lowering the numerical precision of its weights (e.g. FP8 or 4-bit). Operating it becomes markedly cheaper; past a certain point, answer quality drops measurably. for end devices[5]
1.5	24 July 2026	8B, 70B	images as input, audio experimental, thinking mode?Thinking modeA setting in which the model writes intermediate steps before answering. It helps with maths and logic but costs additional tokens and time., context 262,144 tokens
2.0[6]	planned Q1 2027	”an order of magnitude larger”	according to the roadmap, among other things a fairer vocabulary?VocabularyThe fixed list of text pieces a tokenizer knows. Anything not in it as a whole is split into several tokens. for all languages

“8B” stands for 8 billion parameters. Version 1.5 continues the training of version 1.0, with about 4 trillion additional tokens for the 8B and 2 trillion for the 70B.[6] Since then, Apertus has been multimodal?MultimodalA model that can process images, audio or video as input in addition to text.: it reads images, but answers only in text. The team expressly describes the first quarter of 2027 for version 2.0 as a target that may slip.

Lab: what Switzerland has built with Apertus

The ability to do the training itself. The team distributed the training of the 70B model across up to 4,096 GPUs?GPUGraphics processing unit. Language models run almost exclusively on GPUs because they execute very many computations in parallel. of the Alps supercomputer at CSCS?CSCSThe Swiss National Supercomputing Centre in Lugano, part of ETH Zurich. It operates the Alps supercomputer on which Apertus was trained..[4]

Key figure

80 %

Source: CSCS, An Engineering Journey Training Large Language Models at Scale on Alps, arXiv:2604.12973, April 2026; Tech Report Apertus 1.0

CSCS describes the difficulties in a report of its own: memory bottlenecks and the stability of the connections between the machines.[7] The report on Apertus 1.0 lists 103 names, and the report on version 1.5 lists 138 from eight institutions.[4, 6]

Weights that others build on. Because the 8B model already existed, the Apertus Mini family was cheap to make. A team around Martin Jaggi (EPFL) and Dan Alistarh (IST Austria) derived models with 0.5, 1.5 and 4 billion parameters from it by distillation?DistillationA small model learns from the outputs of a large one. This produces cheap, small models with part of the large model's abilities.. The whole family needed less than 12 per cent of the computing effort that the 8B model had cost in pretraining?PretrainingThe first and most expensive training phase, in which a model learns language and knowledge from very large amounts of text..[5]

A narrow base. According to SRF, the first version was worked on by “lediglich zwei Leute Vollzeit, unterstützt von ein paar Dutzend Forschenden” (only two people full time, supported by a few dozen researchers).[3] Who pays for the project and what Switzerland buys politically↗Understanding Apertus · Part 2Apertus and Switzerland: values, languages, sovereigntyAnyone using a language model in Switzerland pays more depending on the national language: with Apertus, the same Federal Constitution needs 60 percent more tokens in Romansh than in English. In addition there is a public charter for the model's behaviour and funding until 2028. All of it can be looked up or measured.Read article with it is examined in the second part of the series.

What is public at Apertus and not at others

For language models, “open” stands for two different things. Open weights?Open weightModels whose weights are publicly available and may be self-hosted (e.g. Apertus, DeepSeek, Llama). A precondition for sovereign operation without vendor lock-in. means: you may download and run the finished model. Open source?Open source (AI)More than open weights: the training code and details of the training data are disclosed too, under a free licence. The Open Source Initiative published a definition for this in 2024. The Swiss Apertus model meets it; many models marketed as "open" only partially do. additionally means that how the model came about can be traced. Why this difference matters for the sovereignty of a company↗Sovereign AI · Part 1What sovereign AI actually means for Swiss companiesIf you sell discretion, you have a problem the moment a client asks where your data goes when you use AI. Sovereign AI answers that question, and the degree of control can be set like a dial.Read article is covered in our series on sovereign AI.

Disclosed	Apertus	OLMo 3	Llama 4	Qwen3	Gemma 3
Weights	yes	yes	yes	yes	yes
Licence	Apache 2.0?Apache 2.0 licenceA free software licence. It permits use, modification and commercial use without requiring you to publish your own code.	Apache 2.0	custom	Apache 2.0	custom
Training data	yes[4]	yes[8]	no	no	no[9]
Data pipeline	yes	yes	no	no	no
Intermediate training checkpoints	yes	yes	no	no	no
Retroactive opt-outs	yes	only at crawl time	not documented	not documented	not documented
Protection against verbatim memorisation?MemorisationA language model learns training texts so closely that it can reproduce passages verbatim. For copyrighted or personal texts this is a legal risk.	yes	not documented	not documented	not documented	not documented
Charter of values	yes[10]	no	no	no	no

“Not documented” means: the model card and the publication say nothing about it. Whether the protection exists remains open.[11]

With Llama, Qwen and Gemma, you have to take the manufacturer’s word for which data went into training. Meta describes the data in the Llama 4 model card as “publicly available, licensed data and information from Meta’s products and services”.[11] For Apertus and OLMo, this can be traced through the published pipeline.[4, 8] How the retroactive opt-outs and the protection against verbatim memorisation work is described in the next section.

Lab: which data was allowed into training

Blocks respected retroactively too. In a file called robots.txt?robots.txtA file on a website in which the operator specifies which crawlers may fetch which pages. AI providers can honour it; they are not legally required to everywhere., a website specifies which crawlers?CrawlerA program that automatically fetches and stores websites, for instance to collect training data for AI models. may read its pages. Many websites now block the crawlers of AI firms, such as OpenAI’s GPTBot or Google-Extended. According to the report, training datasets usually respect such blocks only at the moment of the crawl. What was collected earlier stays in the dataset.[4]

In January 2025, the Apertus team queried the robots.txt of two million domains: the one million with the most English text and the one million with the most text in other languages. If a domain blocked an AI crawler, the team removed its content from all crawls from 2013 to 2024. GPTBot was blocked most often. In the English dataset FineWeb-Edu, 4.5 trillion of 4.9 trillion tokens remained as a result, and in the multilingual FineWeb-2, 45 of 47 trillion. The team puts the loss at about 8 per cent of the English and 4 per cent of the multilingual data.[12]

Whether a model with less web data knows less was examined by the three project leads Martin Jaggi, Antoine Bosselut and Imanol Schlag together with four other researchers. They call the performance difference between training with and without blocked data the “Data Compliance Gap”. For models with 1.5 billion parameters, it was close to zero in general knowledge. In specialist fields such as biomedical research, the texts of large publishers were missing, and the scores dropped. The study was published in 2025 at the COLM conference.[12]

Verbatim reproduction suppressed. Language models can memorise training texts and later output them verbatim, for example a whole newspaper article. Specialists call this memorisation. Apertus was therefore trained with the Goldfish loss?Goldfish lossA training method against verbatim memorisation: a small share of tokens is hidden during learning so that the model cannot reproduce long passages exactly.. This method hides 2 per cent of the tokens during learning, at the same positions on every repetition of a text. Because the model lacks these pieces, it cannot reproduce long passages without gaps. According to the report, this also works for large models and for texts that occurred 128 times in training.[4]

Personal data and toxic content. Before training, a pattern search replaces email addresses, IP addresses and IBAN numbers with placeholders. The report itself calls this “best-effort”. The filter does not search for names or home addresses. In nine languages, including German, French and Italian, the team also removed the 5 per cent of documents that a classifier?ClassifierA small model that assigns texts to a category, such as "sensitive topic" or "harmless". In training it serves as a filter or router. rated as most abusive or discriminatory. Code, mathematics and education datasets were exempt from this filter.[4]

Only clear licences in fine-tuning. In fine-tuning, a model learns to follow instructions. For this, the team admitted only datasets whose licence permits redistribution and commercial use, such as CC-BY or Apache 2.0. Data with a non-commercial licence (NC), with a share-alike licence (SA) or without a clear licence was dropped. The team measured what this costs in an experiment. It fine-tuned an intermediate checkpoint of the 8B model, taken after 10 trillion tokens, twice: with the widely used training mixture Tulu 3 and with its licence-filtered version.[4]

Key figure

−5.8 %

Source: Tech Report Apertus 1.0, arXiv:2509.14233, Table 10

Test	without licence filter	with licence filter	Change
Average over 13 tests	0.443	0.417	−5.8 %
Knowledge test with worked solution (MMLU-CoT)	0.513	0.253	−51 %
Knowledge about Switzerland (SwitzerlandQA)	0.590	0.579	−1.9 %
Truthfulness of answers (TruthfulQA MC2)	0.486	0.518	+6.6 %

Across all multilingual tests, the average fell by 4.3 per cent. The team writes that it accepts this trade-off, “as necessary for responsible open-source model development”.[4]

Country: how a Swiss legal view found its way into the model

For the legal basis, the report points to an analysis of training on third-party content under Swiss law. It was written by David Rosenthal and Livio Veraldi of the Zurich law firm VISCHER. Both are listed in the report as advisers to the project.[4] The analysis appeared in 2025 in the journal Jusletter IT.[13]

The data rules follow an opt-out approach. Objections to the use of content are respected, even after the fact. Anyone who does not object is deemed to agree: if there is no robots.txt, the team uses the data. Moreover, only the domains with the most text were queried. A small website that blocks AI crawlers may therefore still have ended up in training. From an adviser’s point of view, the approach is nonetheless progress. A legal view that otherwise sits in expert opinions can be checked here against code and data.

In the EU, disclosure is already mandatory. The EU AI Act↗Law & Regulation · Part 5EU AI Act Switzerland: role first, then dutiesMost Swiss companies owe the EU AI Act almost nothing. The provider role gets expensive, that is, whoever ships an AI tool under its own name. Whoever lands in it usually notices too late.Read article requires in Art. 53 that providers of general-purpose models?General-purpose AI model (GPAI)An AI model capable of many different tasks and built into a wide range of applications, such as GPT or Llama. The EU AI Act regulates such models separately, addressing their providers. publish a summary of their training data, following a template from the European Commission. Researchers at Trinity College Dublin and the Mozilla Foundation reviewed five published summaries for Tech Policy Press in March 2026. Apertus received the top grade on every criterion. From OpenAI, Google and xAI, no summary was available at the time.[14]

A publicly funded Swiss institution has thus shown that the EU’s transparency rule can be met. For the debate on how Switzerland intends to regulate training on third-party content, Apertus also supplies measured costs. Rights holders and model builders can both draw on them. This assessment is not legal advice.

Lab, country, operations: how the series reads Apertus

Every decision of the project can be read on three levels. Each part of the series goes through them in this order.

Level	Question	Example from this part
Lab	What did the team build, research and measure?	retroactive opt-outs, Goldfish loss, licence filter with −5.8 %
Country	What does this mean for Switzerland, politically and legally?	a Swiss legal view on training, checkable against code
Operation	What does this mean for a company that uses AI?	screening questions for every model provider

This first part describes what Apertus is and how it was built. The second asks who funds the project and what independence it brings Switzerland. The third examines which tasks the model suits in operation, where it runs and what it costs there.

What this means for your company

From an adviser’s point of view, the licence experiment changes how model comparisons should be read. Models that score better in tests are better in part because they learned from material that Apertus deliberately declines to use. How large this share is for a specific model can only be measured if its training data is public. For which tasks Apertus is nonetheless the right choice in operation is examined in the third part of the series.

If the model reproduces a name or a phone number despite the filters, your own application has to catch that. The team accepts deletion requests from data subjects at llm-privacy-requests@swiss-ai.org.[4]

The Apertus reports are a useful yardstick even if you use a different model. For a first conversation with a provider, four questions are enough:

1. Is there a published summary of the training data under the EU AI Act, and where can it be found?
2. Were website blocks respected, including retroactively for older crawls?
3. What prevents the model from reproducing training texts verbatim?
4. Which personal data was removed before training, and does anyone filter the outputs?

Apertus answers all four publicly. Get the provider’s answers in writing. Ten questions with the answers from Apertus as a comparison are included as a worksheet in the PDF.

Sources

Do you know what the language model your company uses was trained on?

With the Swiss model Apertus from ETH, EPFL and CSCS, you can check. Of five open models compared, only OLMo 3 allows the same. With Llama 4, Qwen3 and Gemma 3, only the weights are open (model cards, October 2026).

The price is in the team's report. On mathematics and code, Apertus trails OLMo 3 (report on version 1.5). The report on version 1.0 also states what the team left out of training. It removed content from websites that block AI crawlers from all crawls since 2013. That cost about 8 per cent of the English web data. In an experiment with the 8B model, it left out data with a non-commercial or unclear licence in fine-tuning. Across 13 tests, performance fell by 5.8 per cent as a result, and by half in one knowledge test with a worked solution.

Our conclusion: part of the lead other models have comes from material that Apertus has declined to use. Anyone comparing models should know that. It can only be checked where data and pipeline are public.

In the article: the model family up to the planned version 2.0, the legal basis of the Zurich law firm VISCHER, the review under the EU AI Act by Trinity College Dublin, and a worksheet with ten questions for every model provider. Link in the comments.

#Apertus #AI #Copyright #Switzerland #DataSovereignty