Who trains the models, who serves them fast, what they cost, how quick they run, and whose law governs your data when you call them. A snapshot built to make provider tradeoffs legible, not to sell any one of them.
✓ verified confirmed 3-0 against primary sources⚠ snapshot true mid-2026, decays fast◆ context architectural background, not independently cited
00 The six things to hold
1 · Three layers, not one
Labs train weights. Inference providers serve them. Chips run underneath. Model choice and serving choice are independent decisions.
2 · Speed is a serving property
Cerebras/Groq hit ~1,000–2,000 tok/s on open weights, ~10× typical GPU serving. They train nothing.
3 · China is frontier-adjacent
GLM-5.2 sits ~4.6 pts behind the top model on the intelligence index, and DeepSeek is 20–100× cheaper than Western frontier.
4 · Jurisdiction follows the route
A "Chinese model" run on a Frankfurt server is under EU law. The weights are stateless; the server is where your data lives.
5 · Open ≠ open source
"Open weights" gives you the model file, rarely the training data or code. The license still constrains you.
6 · Everything here decays
Prices, ranks and tok/s move weekly. Treat every number as a July 2026 reading, not a constant.
01 The three layers
The single most useful decomposition. When someone says "we use Groq," they've told you nothing about which model they run, and vice versa. These are orthogonal.
A request passes downward: pick a model, pick who serves it, land on some silicon
The structural tell ✓: Cerebras lists 6 models, Groq lists 11 — every one is an open-weight model trained by someone else (GLM from Zhipu, Gemma from Google, gpt-oss from OpenAI, Llama from Meta, Qwen from Alibaba). A pure inference provider owns the speed layer, never the intelligence layer. That's why you can't buy Opus or Gemini "on Groq" — closed weights never leave their lab's own serving stack.
02 Inference physics — where the real tradeoffs live
Skip the "different companies" framing. The interesting question is why an LPU hits 2,000 tok/s, what it costs to get there, and when it's the wrong tool.
Why custom silicon is fast ◆ context
Autoregressive decode is memory-bandwidth-bound, not compute-bound. Every generated token streams the entire active weight set through the ALUs once. So throughput ≈ memory-bandwidth ÷ active-parameter-bytes. The whole game is keeping weights close to the compute.
A GPU parks weights in HBM (~3–8 TB/s on H100/H200). Custom silicon keeps them in on-chip SRAM, which is roughly an order of magnitude faster. Groq's LPU is SRAM-resident and deterministic, so there's no HBM round-trip per token. Cerebras' wafer-scale engine fits an entire model on one dinner-plate-sized chip; the weights never leave the die. That is the mechanism behind the sub-second time-to-first-token and the four-figure tok/s. ◆(bandwidth figures are background knowledge, not from the cited research)
⚑
The catch that explains their model menu: SRAM is tiny per chip, so holding one model takes many chips wired together — capex-heavy, and it favours smaller-active-parameter models. That is precisely why Groq and Cerebras serve MoE and mid-size open-weight models (gpt-oss, Gemma, GLM, Qwen) rather than dense frontier giants. The hardware picks the model class.
Open weights → a commoditized serving market ✓ verified
Because open weights can run anywhere, the same model becomes a price/latency auction. On Llama 3.3 70B across 15+ providers, Artificial Analysis measured an 18× spread on speed and an 8.6× spread on price — same weights, same output quality.
Dimension on Llama 3.3 70B
Fastest / cheapest
Slowest / priciest
Spread
Output speed
Groq 311 tok/s(within AA's set)
DeepInfra ~16.9 tok/s
18×
Blended price / 1M
DeepInfra $0.12
Scaleway $1.05
8.6×
→
Build implication: for a fixed open model, which provider you pick moves latency and cost more than most model swaps do. Choose the model for capability, then shop the serving layer separately. (Caveat: "Groq fastest" holds only within the provider set AA listed for that page — Cerebras serves 70B-class Llama faster but wasn't on it. Always scope speed claims to a model version + provider set.)
03 Open vs closed — the license mechanics
"Open" is three different things people conflate. The distinctions have real legal and cost consequences.
Term
What you actually get
Examples
Catch
Closed / API-only
An endpoint. No weights, ever.
Claude, GPT-5, Gemini, Grok
Can't self-host, can't audit, locked to the lab's regions
Open weights
The model file. Run it anywhere, fine-tune it.
Llama, GLM, Qwen, DeepSeek, gpt-oss, Gemma, Kimi, Mistral
Rarely ships training data or code; license may restrict use
True open source (OSI)
Weights + training code + data recipe, permissive license
Rare at frontier scale
Almost nobody at the top does this fully
The licenses that matter
License
Freedom
Who uses it
MIT / Apache 2.0
Maximally permissive — commercial, modify, redistribute, no user cap
DeepSeek R1 (MIT) ✓, Gemma (Apache), many Qwen/Mistral releases
Llama Community License
Free to use commercially until you cross ~700M monthly active users, then you need Meta's permission
Meta Llama family
Custom / source-available
Weights public, but terms vary (non-compete, no-training-competitors clauses)
Various Chinese + Western releases
What open weights buy you: self-hosting (data never leaves your infra), price competition across providers, fine-tuning, and freedom from a single lab's uptime and region limits. What they cost you: you own the ops, the eval, and the safety tuning — and the very top of the capability curve is still closed.
04 Provider profiles
Flagships as of mid-2026. Open/closed badge, positioning, one sharp tradeoff each. China labs are first-class here, not a footnote.
AnthropicClosed
Claude Opus 4.8 · Sonnet 5 · Haiku 4.5 · Fable 5
Tops the intelligence index. Strongest at coding + agentic work. Weakness: no first-party EU data residency (US-only serving).
OpenAIClosed
GPT-5.4 · GPT-5.5 · (5.6 family emerging)
Broadest ecosystem, 1M context, best EU data-residency story of the closed labs. Also ships gpt-oss open weights.
Google DeepMindClosed
Gemini 3.1 Pro · Gemma (open)
Huge context, deep GCP integration, TPU-served. Open Gemma line feeds Cerebras/Groq. Index score slipped mid-2026.
MetaOpen weights
Llama family
Lit the open-weight wave. Widest multi-provider serving market. Community license caps the mega-scale users.
MistralOpen weights
Mistral / Mixtral line
European (Paris), Apache releases, GDPR-native positioning. Strong efficiency, smaller than the US frontier.
xAIClosed
Grok family
Fast follower, real-time X data access. Positioned on freshness + fewer refusals than heritage on capability.
DeepSeekOpen weights
V4 Flash · V4 Pro · R-series
The cost disruptor. MIT-licensed reasoning, MoE, ~20–100× cheaper than Western frontier. First-party API is China-hosted (see §06).
Zhipu / Z.AIOpen weights
GLM-5.2 (GLM-4.7 on Cerebras)
Leading open-weight model on the intelligence index (6th overall, 51.1%). Coding-strong, served on Cerebras at ~1,000 tok/s.
Alibaba (Qwen)Open weights
Qwen3.7 Max · Qwen3 line
Deep, wide family (dense + MoE + multimodal). On Groq and Bedrock. The workhorse open Chinese line.
Moonshot (Kimi)Open weights
Kimi K-series
Long-context specialist, agentic strengths. Newer to Western serving; check availability per provider.
05 Price, speed, intelligence
Token price — cheapest to frontier ⚠ snapshot
Model
$/M input
$/M output
Context
Note
DeepSeek V4 Flash✓
$0.14
$0.28
1M
the floor — 20–100× under frontier
DeepSeek V4 Pro ✓
$0.435
$0.87
1M
near-frontier, still tiny
Claude Haiku 4.5
$1
$5
—
cheap Western closed
Claude Sonnet 5
$2
$10
—
⚠ intro rate, reverts to $3/$15 after Aug 31 2026
GPT-5.4
$2.50
$15
1M
mid-tier frontier
Claude Opus 4.8
$5
$25
—
premium frontier
GPT-5.5
$5
$30
1M
premium frontier
Capability — Artificial Analysis Intelligence Index ⚠ volatile
Model
Index
Note
Claude Opus 4.8
~55.7
top reference point (exact ordering vs GPT-5.5 was contested)
GLM-5.2 (Zhipu)
51.1
leading open-weight model, ~4.6 pts off the top, rank ~6
Gemini 3.1 Pro
46.5
was ~57 in Feb–Mar 2026 — shows how fast this moves
Qwen3.7 Max
46.0
Alibaba open
MiniMax M3
44.4
another Chinese contender
✦
The open-weight gap is small and structural ⚠: open models have trailed US frontier by a consistent 3–6 months for 18+ months (OpenRouter). Epoch AI independently measures ~4 months, and notes it slightly widened from 3 to 4 — so "keeping pace" is right, "closing the gap" is not yet. Either way, close enough that open/Chinese labs are a real choice, not a compromise.
06 Which country's law governs your data
The question that actually decides what you're allowed to build with, from Switzerland/EU. And the one place the "Chinese model" panic is mostly misdirected.
The core principle
Jurisdiction follows the route, not the model.
"Chinese model" tells you nothing about which law applies. What matters is whose servers process the request. The same GLM or Qwen weights: hit the first-party Chinese API and you're under Chinese law; hit AWS Bedrock in Frankfurt and you're under EU law. Weights are stateless. The server is where your data lives.
Where each route puts your data ✓ verified
Route
Processed in
Retention / training on inputs
EU/Swiss-clean?
OpenAI API (EU Projects)
Europe, in-region
Zero retention on EU Projects; in-region GPU inference since Jan 16 2026
✅ business/new-Projects tiers
Anthropic Claude API
US only
Not retained by default; no training w/o permission; ZDR available
❌ no first-party EU residency
Google Gemini (Vertex)
selectable, incl. EU
Enterprise: no training on inputs
⚠ EU regions exist (not re-verified this pass)
DeepSeek first-party API
China (PRC)
Stores personal data in PRC & trains on inputs (opt-out only)
❌ the genuine trap
Chinese weights on AWS Bedrock (Frankfurt / Zurich EU profiles)
EU/EEA only
AWS terms; cross-region routing stays in EU/EEA (immutable)
✅ the clean way to run DeepSeek/Qwen
Groq
US + Helsinki (EU)
Not retained by default
✅ EU data center since Jul 2025
Together AI
25+ cities, EU capacity
Offers data residency, GDPR
✅
Fireworks / DeepInfra
US-centric
Zero retention by default (no prompt logging, no training)
~ partial
Cerebras
~85% US
—
❌ mostly US (EU buildout announced)
Three things to memorize
The DeepSeek trap
The first-party API stores data in China and trains on your inputs by default. That's why Italy (Garante) and South Korea banned it✓. "We can't use DeepSeek" means the app, not the weights.
The Western-host play
To use a Chinese model GDPR-cleanly, run the open weights on a Western host. DeepSeek + Qwen are on Bedrock Frankfurt; Zurich/EU profiles route only within EU/EEA. Data never touches a Chinese server.
The Anthropic surprise
Claude's first-party API has no EU data residency — it processes in the US. For Claude on EU soil you go through Bedrock or Vertex, not Anthropic directly. Counterintuitive given its enterprise reputation.
⚑
Two honesty flags. (1) The reassuring line that cross-region inference "holds data in memory only, never persisted in the destination region" was refuted 0-3 — don't assume the destination region is stateless; the safe claim is only that EU profiles keep routing within the EU. (2) First-party PIPL/hosting detail is confirmed for DeepSeek only; for GLM, Qwen and Kimi assume the same shape (China-hosted first-party, safe via Western hosts) but treat it as inferred, not verified.
07 Intuition — what is 1M tokens?
The conversion runs the opposite way most people guess: 1 token ≈ 4 characters ≈ 0.75 words. So 1M tokens is a lot more than it sounds.
~750k
words
~4M
characters
~8
average novels (or 1.5× the LOTR trilogy)
~2,500
book pages
75–100k
lines of code (denser: ~8–10 tok/line)
Task-level anchors (input side)
Task
≈ tokens
Summarize a 300-page book
~120k
Analyze a 50-page contract / report
~30k
A long coding / chat session's full context
50k–200k
Ingest an entire medium repo
500k–1M+
Tie it to money — reading a whole ~100k-word novel (~133k tokens)
Model
As input
As output (generating that much)
DeepSeek V4 Flash
~2¢
~4¢
GPT-5.4
~33¢
~$2.00
Claude Opus 4.8
~67¢
~$3.30
→
Two meanings, one number. "1M tokens" as a context window (Gemini, GPT-5) = how much you can stuff in at once (~750k words). "1M tokens" as a pricing unit ($/M) = the billing meter. Same figure, different axis. And note output is the expensive direction — that ~80× Opus-vs-DeepSeek output gap is the whole open-vs-frontier cost story in one number.
08 Key moments
⚠ Pre-2025 dates are well-known but weren't independently re-verified in this research pass; the Jan 2025 DeepSeek entry onward is confirmed.
2020
GPT-3
Scaling laws go mainstream; the API era of LLMs begins.
Nov 2022
ChatGPT
The consumer moment. Fastest product to 100M users; LLMs enter everyone's vocabulary.
Mar 2023
GPT-4
The capability jump that set the frontier bar for two years.
2023
The Llama open-weight wave
Meta ships usable open weights; a whole ecosystem (fine-tunes, quantization, local serving) blooms.
Late 2024
Reasoning models (o1)
Test-time compute: models "think" before answering. A second scaling axis opens.
Jan 20 2025
The DeepSeek moment ✓
DeepSeek R1: MIT-licensed, 671B-param MoE (37B active), frontier-adjacent reasoning at ~27× under OpenAI o1. Proved cheap + open + capable could coexist, and that export controls don't stop it. Market-moving.
2025 → mid-2026
Multi-polar frontier
GPT-5.x, Claude 4.x + Fable 5, Gemini 3.x, and a wall of Chinese open weights (GLM-5.2, Qwen3.7, Kimi). Custom-silicon serving (Cerebras/Groq) turns speed into a product axis. Export controls tighten (Singapore/Malaysia channel closed May 31 2026).
09 Glossary — the sharp bits
Skipping the basics you know. Kept: the terms that actually come up in tradeoff conversations.
Mixture of ExpertsMoE
Only a subset of parameters (the "active" ones) fire per token. DeepSeek R1 is 671B total but 37B active. This is why the economics work: you pay compute for active params, but capacity for total. It's also why custom-silicon serving loves MoE — small active footprint fits in SRAM.
Reasoning / thinking models & test-time compute
The model generates hidden intermediate "thinking" tokens before the answer, trading latency and output cost for accuracy on hard problems. A second scaling axis: instead of a bigger model, spend more compute at inference. Why o-series and R-series are pricier per task than their token price suggests.
Distillation
Train a small "student" model on a large "teacher's" outputs. How frontier capability trickles down into cheap, fast small models — and a live geopolitical sore point (who distilled whose outputs).
Quantization FP16 → FP8 → FP4
Store weights at lower numeric precision to cut memory and boost speed. FP8 is now near-lossless in practice; FP4 starts costing measurable quality. When a provider serves a model cheaper and faster, it's often quantized harder — worth checking if you care about the last few points of quality.
Prefill vs decode · tok/s vs TTFT
Prefill = processing your prompt (compute-bound, parallel) → sets time-to-first-token. Decode = generating the answer one token at a time (memory-bound, serial) → sets tok/s. Interactive/agentic apps care about TTFT + tok/s; batch jobs care about $/token. Different silicon wins each.
KV cache
The stored attention state for tokens already processed, so each new token doesn't reprocess the whole context. It grows with context length and eats memory bandwidth — the hidden reason long contexts get slow and expensive, and a big part of what serving infra optimizes.
Speculative decoding
A small draft model proposes several tokens; the big model verifies them in one pass. Free speed when the draft is right. One of the tricks GPU providers use to close the gap on latency-optimized silicon.
Active vs total parameters
The number that decides serving cost and hardware fit. Ask "how many active params?" not "how big?" — a 671B MoE with 37B active serves more like a 37B model than a 671B one.
10 Sources & confidence
Two adversarial research passes: 49 sources fetched, 213 claims extracted, 50 fact-checked with 3-vote verification. Primary sources (teal) weighted over aggregators. Below are the load-bearing ones.
Intelligence Index rankings (aggregator, cross-checked)
⚑
Killed in verification (don't repeat): the exact Opus-vs-GPT-5.5 index ordering (0-3); Groq's "10× faster & cheaper than GPUs" CEO line (real independent figure ~3–4× on equivalent models, ~10× only vs typical GPU serving); a specific DeepSeek V4 Flash spec sheet (0-3); the cross-region "memory-only, never persisted" claim (0-3).