The Open-Weight Frontier Gap

Sources#

• Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident
• Gemma 4 Technical Report
• Inkling: Our Open-Weights Model
• Kimi K3 Model Card
• Ramp's latest data on China vs. the American AI Labs
• Security incident disclosure — July 2026
• Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning

Summary#

Table 4 of the Gemma 4 report is a snapshot of the open-weight landscape as of June 19, 2026, measured on Arena Text — blind side-by-side human preference, Elo-rated. It is the only externally adjudicated number in a document otherwise full of self-reported benchmarks, and it is more informative than any of them.

Rank	Model	Elo	Open	Type	Params / active
1	Claude Fable 5	1508	no	—	—
~15	GLM 5.1	1475	yes	MoE	744B / 40B
29	MiMo V2.5 Pro	1466	yes	MoE	1T / 42B
34	Kimi K2.6	1460	yes	MoE	1T / 32B
36	DeepSeek V4 Pro Thinking	1458	yes	MoE	1.6T / 49B
43	Gemma 4 31B	1451	yes	Dense	31B
57	Qwen 3.5 397B-A17B	1444	yes	MoE	397B / 17B
61	Gemma 4 26B-A4B	1438	yes	MoE	26B / 4B
157	Gemma 3 27B	1366	yes	Dense	27B

Three readings#

The gap is small in Elo and enormous in parameters. The best open model trails the best closed model by 33 Elo; Gemma 4 31B trails it by 57. But GLM 5.1 buys its 24-point lead over Gemma with 744 billion parameters against 31 billion — a 24× ratio, and it activates 40B per token, more than Gemma's entire dense model. "Open weights have nearly caught up" and "open weights at the frontier require a datacenter" are both true, and they are the same sentence read from different ends.

Frontier-open means MoE; Gemma is not playing that game. Every open model above Gemma 4 31B is a Mixture-of-Experts in the 397B–1.6T range. Gemma's own summary — "the leading dense open model on the leaderboard" — is a precisely scoped claim, and the scoping is the point. The family's stated target is "varied hardware environments" and "edge deployment." It competes on inference efficiency, where a 0.8 GB quantized E2B is a category the 1.6T models cannot enter at all. Two open-weight strategies have separated: approach the frontier with sparsity, and approach the device with efficiency.

And the sparsity side now has a documented training method. The GLM 5.1 at the top of this table (744B/40B) is the immediate predecessor of GLM-5.2 (750B-A40B), which the SAO paper reports training with asynchronous single-rollout RL. So the corpus now has both ends of the frontier-open MoE story: where these models land (this page) and how they are trained to get there (Asynchronous RL for LLMs). The same paper's Table 1 also shows GLM-4.7 beating GPT-5 High and Claude-Sonnet-4.5 on three of four math benchmarks — the capability-side counterexample to Gemma's efficiency-side positioning, on measured benchmarks rather than Arena Elo.

A third strategy arrives: customize, don't compete (July 2026). Inkling — TML's 975B/41B-active MoE, released with full weights — lands squarely in the frontier-MoE parameter class but declines the frontier race: "Inkling is not the strongest overall model available today, open or closed" is TML's own sentence, and its vendor tables confirm it (HLE text 29.7 vs GLM 5.2's 40.1; Terminal Bench 63.8 vs 82.7). Its claimed edge is being the best base for fine-tuning — multimodal, token-efficient via a controllable effort dial, hosted on TML's Tinker platform. So the two-strategy picture this page drew (approach the frontier with sparsity vs. approach the device with efficiency) gains a third axis: approach the fine-tuner with adaptability. It also breaks the pattern noted below: a Western lab shipping a ~1T-class open MoE (vendor-claim; no Arena placement yet).

The sparsity pole goes to 2.8T, and the agentic gap looks wider than the chat gap (July 2026). Kimi K3 (vendor-claim) is a 2.8T-total / 104B-active MoE — Moonshot calls it "the world's first open 3T-class model," nearly double the 1.6T DeepSeek entry in the table above and 2.9× K2.6's own 1T. It is the strongest available restatement of this page's first reading: the open side keeps closing the capability gap by spending parameters, at ratios the closed side never has to disclose.

Two things it adds that the Arena snapshot could not. First, numbers on the agentic axis. The card cites two third-party Artificial Analysis Elo measures of long-horizon work: GDPval-AA v2, where K3 scores 1686 against Fable 5's 1747 — a 61-Elo gap — and AA-Briefcase, 1548 vs 1583, a 35-Elo gap. Both are wider than the 33-Elo chat gap this page opened with, and the wider one is on the more economically-framed benchmark. Second, a shape to the residual gap: across 45 benchmarks K3 tops 14 rows, and it tops them almost entirely on retrieval, MCP/tool orchestration and document vision (BrowseComp 91.2, MCPMark 94.5, OmniDocBench 91.1) while trailing on hard reasoning (HLE-Full 43.5 vs 53.3) and the hardest long-horizon coding (FrontierSWE 81.2 vs 86.6, OSWorld 2.0 58.3 vs 66.1).

Treat all of it as self-reported: the Elo figures are third-party but selected and transcribed by the vendor, K3 runs on Moonshot's own Kimi Code harness while rivals are quoted at their best across harnesses, and Moonshot itself flags that Fable 5 was hitting fallbacks during two of the coding evaluations (Compute-Controlled Benchmarking). No Arena placement for K3 exists yet — when one lands it will be the first clean test of whether a 2.8T open model narrows the 33 points.

DeepMind's MoE loses to DeepMind's dense model. Gemma 4 26B-A4B (Elo 1438, rank 61) sits 13 Elo below Gemma 4 31B (1451, rank 43), on human preference, from the same lab in the same release — while every larger open model in the table proves MoE scales. On static benchmarks the MoE is close to the dense model (MMLU Pro 82.6 vs 85.2, AIME 88.3 vs 89.2) and sometimes ahead (τ²-airline 76.0 vs 75.0), but human raters prefer the dense 31B. The report does not remark on this. If the effect is real, it suggests sparsity's returns arrive at scales far above 26B, or that active-parameter count (3.8B) governs the qualities Arena raters respond to.

Why the human-preference number is the trustworthy one#

The rest of the report is Google measuring Google. Arena is blind, third-party, human-rated, and confidence-intervalled (Gemma's ±8). It also disagrees with the static benchmarks in a legible way: Gemma 4 31B is claimed to rival "larger, frontier open models," and on Arena it does — 1451 against DeepSeek V4 Pro's 1456, inside the combined error bars. That is a real result, and it is real precisely because Google didn't grade it.

The same table quietly cites Claude Fable 5 at rank 1 — a third-party corroboration of a model whose own wiki page rests entirely on Anthropic's vendor-claim announcement, and whose head-to-head benchmark table was published only as an untranscribed image. A competitor's leaderboard is better evidence for Fable 5's standing than Fable 5's launch post.

What the table does not control for#

Arena Elo carries no compute budget. Gemma 4's entries are thinking-mode models; the table doesn't say at what thinking budget they were served, nor what the closed models spent. Per Compute-Controlled Benchmarking, a preference score without a budget has the same defect as a benchmark score without one — it just hides it behind human judgment instead of a number. The 33-Elo open-versus-closed gap could be a capability gap, an inference-spend gap, or both.

The demand side: who is actually buying this (July 2026)#

Everything above is supply-side — what open models score, at what parameter count, on whose harness. Ramp's July 2026 AI Index letter (Ara Kharazian, empirical) is the first measurement in this vault of whether US businesses are paying for any of it, read off corporate-card and bill-pay records. Its answer is narrow but precise, and it is a more falsifiable claim than "China is catching up."

The proxy. Ramp has no per-model visibility, so it counts firms paying model-serving and inference platforms — vendors that resell hundreds of models — as its stand-in for open-source and Chinese model use. 5.8% of AI-spending businesses used one in June 2026, up from 4.5% in January. The recovered monthly series (the letter reports only two points) puts that in a longer frame: 1.03% in July 2023 → 5.77% in June 2026, a 5.6× rise as a share of AI adopters, with the slope roughly doubling in 2026 (+1.24pp across all of 2025, +1.30pp in the first five months of 2026). Rising and small are both true.

The proxy is loose in both directions, and the second direction matters for this page. It over-counts: a serving platform sells Llama, Mistral and Qwen alongside GLM and Kimi, so American open weights land in the same bucket as Chinese ones. And it under-counts the case this page has spent the most words on — self-hosting is invisible to it. The Hugging Face forensics below ran nvidia/GLM-5.2-NVFP4 on HF's own endpoints; a firm that downloads open weights and serves them on its own GPUs pays no model-serving vendor and does not appear here at all. Treat 5.8% as a loose upper bound on paid Chinese-model access, not a measurement of open-weight use.

The finding worth keeping: adoption is additive, not substitutive. Among the firms using model-serving platforms, 85.8% also pay OpenAI, 93.2% also pay Anthropic, and 96.4% pay at least one directly. Decomposed (vault arithmetic on Ramp's three numbers): 82.5% pay both labs, and only 3.6% pay neither.

That is stronger than it first looks, because the right comparison is not 100% but the base rate. Rebasing the letter's vendor-share chart onto AI-spending businesses (also vault arithmetic, not stated by Ramp): in June 2026 71.8% of AI-spending businesses paid OpenAI and 77.2% paid Anthropic. So the firms buying open/Chinese model access are 14pp and 16pp more likely to pay the American labs than the average AI spender — the least substitution-shaped result the data could have produced. They are also, by an order of magnitude, the biggest AI spenders in the base: median $248.41 per employee per month against $10.59 for the typical AI-spending business (23.5×). Cheap models are showing up as more AI spend, not less.

One more line the letter never mentions while arguing about Chinese models: DeepSeek's own direct-payment share peaked at 0.23% of businesses during the January 2025 R1 news cycle, fell for a year, and reaches only 0.29% in June 2026 — 1/20th of the serving-platform proxy.

What it does not settle. Additive-today is also the shape an early substitution curve has: a firm evaluates a cheaper model on some tasks while keeping its frontier contracts, and the 96.4% only falls once routing share moves enough to cancel a subscription. The instrument sees whether a vendor is paid, never how much traffic each model gets, so a firm that moved 80% of its tokens to GLM and kept a $20 seat still counts as "also uses Anthropic." Kharazian's own reading is the honest one — the growth "reflects a real weakness for American model companies" on price, and the absence of broad price cuts is the corroborating signal that share has not actually moved. And the sample is Ramp's own VC-forward-skewed customer base, from a vendor with a commercial interest in owning this dataset (see Telemetry vs. Survey Measurement); the direction is more defensible than the level.

Connections#

• Gemma 4 — the source of the table; the leading dense open model
• Claude Fable 5 — rank 1, and cited here by a competitor rather than by its vendor
• Inference Efficiency as Capability — the axis Gemma competes on instead of scale
• Compute-Controlled Benchmarking — an Elo score without a budget is still a score without a budget
• Open-Weight Elicitation Irreversibility — what "open" costs, once these models carry a thinking mode
• Jagged Intelligence (Ghosts, Not Animals) — the aggregate Elo hides that small Gemmas beat Gemma 3 27B on reasoning and lose on knowledge
• Large-Scale Test-Time Compute — the unnamed variable underneath every cell of the table
• Encoder-Free Early Fusion — one of the levers that lets a 31B dense model contend at all
• Responsible Scaling Policy Evaluations — why the open-weight safety argument here is structural: Gemma 4 sits at rank 43, nowhere near a risk threshold
• Task Time-Horizon Scaling — Arena scores chat preference; whether the open/closed gap survives on long-horizon agentic work is a different measurement
• Google DeepMind — publishes the table, and places itself 43rd on it
• GLM (Z.AI) — GLM 5.1 (top of the table) and its SAO-trained successor GLM-5.2: the capability-side open-weight strategy this page contrasts with Gemma's efficiency-side one
• Single-Rollout Optimization — the RL method behind the GLM MoE line's continued frontier presence; the training-side complement to this landing-place snapshot
• Kimi (Moonshot AI) — K2.6 sits at rank 34 in the table; K3 at 2.8T/104B is the sparsity pole's current extreme and the first open release with agentic-Elo numbers to set beside the chat gap
• Autonomous Intrusion — a fourth reason open weights matter, and the first one that isn't about the leaderboard. This page's three strategies (frontier-by-sparsity, edge-by-efficiency, fine-tunability) are all capability arguments. Hugging Face's July 2026 incident disclosure adds self-hostability as an operational requirement: frontier commercial APIs' safety guardrails refused to process its attack payloads, so the forensics over 17,000+ attacker events ran on GLM 5.2 locally. The relevant property is not that the open model is close on Elo — it is that it is the one that will run on data a hosted model declines. Sharpened 2026-08-03 by HF's technical post-mortem: the refusing APIs are named ("Claude Opus and Fable"), and the replacement was Nvidia's NVFP4 quantization of GLM-5.2 on HF's own endpoints — so the operational requirement was met by a quantized frontier-open MoE, which matters for this page's parameter-vs-capability framing: the datacenter-class model was the only one available for the job, but it did not have to be run at full precision to do it. And the job was not summarization — GLM-5.2 recovered the agent's chunk+XOR+compress scheme and its per-campaign key, surfacing ~4× the secrets a naive scan found. Still a single first-party account, and one authored by the company that hosts the open-weight ecosystem
• Autonomous Defense — where that requirement bites: a hosted-API SOC degrades exactly at the top of the severity distribution
• Firm AI-Spend Intensity and Headcount Growth — the buyer-side instrument behind the section above, and the population it isolates: the model-serving cohort's $248/employee/month is ~7× the high-intensity mean in that page's Ramp × Revelio panel, so the firms buying open/Chinese model access are an extreme tail inside the group that panel measures growing headcount ~10%
• AI Product Economics Maturation — the second demand-side reading, from a survey rather than a payment rail: among ~305 AI-building software companies, DeepSeek 7% / Alibaba 6% / Moonshot 4% against Anthropic 81% and OpenAI 71% (select-all). A cohort far more AI-intensive than Ramp's card base, and the Chinese labs still sit an order of magnitude below the American ones
• Telemetry vs. Survey Measurement — why the 5.8% is a floor-and-ceiling problem rather than a number: a payment-rail instrument sees only vendors that get paid, so self-hosted open weights (this page's whole point about downloadable frontier MoEs) are invisible to it by construction
• Balance-of-Power Superintelligence — the philosophy that underwrites closing this gap from the open side; the August 2026 manifesto confirms Meta's open-weight pause was a pause ("we will resume releasing some open source models soon") and asks that US policy stop restricting American open models on training-data grounds

Open Questions#

• Is the dense-beats-MoE result at 26B robust, or an artifact of one Arena snapshot with ±8 error bars on both models? (The two intervals overlap: 1451±8 and 1438±8.)

• The open MoE giants (GLM, DeepSeek, Kimi, MiMo, Qwen) are overwhelmingly Chinese-lab releases. Gemma is the Western open-weight entry and it targets the device, not the frontier. Is that a strategic choice or a capability constraint? Partially answered (2026-07-22): Inkling is a Western 975B/41B open MoE — so Western labs can and do ship at frontier-open scale — but it self-reports below GLM 5.2 / Kimi K2.6 on hard reasoning and coding and explicitly declines the frontier framing in favor of a customization axis. One release, still consistent with either reading of the remaining gap.

• Arena measures preference on chat. Does the 33-Elo open/closed gap widen or collapse on long-horizon agentic work, where time-horizon rather than response quality governs? Partially answered (2026-07-30): Kimi K3's card cites two Artificial Analysis agentic Elo boards where the best open model trails the best closed one by 61 (GDPval-AA v2: 1686 vs 1747) and 35 (AA-Briefcase: 1548 vs 1583) — both wider than 33, pointing to widen-not-collapse. But the comparison is vendor-selected, harness-asymmetric, and taken against a Fable 5 that Moonshot itself reports hit fallbacks on 35% of one coding benchmark, so the direction is indicative rather than settled.

• Does open/Chinese-model adoption ever become substitutive rather than additive? The falsifiable version: Ramp's 96.4%-of-model-serving-users-also-pay-OpenAI-or-Anthropic figure is published monthly, so a sustained fall in it — or in the 82.5% who pay both — while the 5.8% serving-platform share keeps rising is the signature of displacement. Absent that, rising serving-platform use is a story about firms buying more AI, not about the American labs losing share. Trigger: the monthly Ramp AI Index, and any broad frontier-price cut (Kharazian names the absence of one as evidence share has not moved).

Sources#

• Ramp's latest data on China vs. the American AI Labs — Ara Kharazian, Ramp's latest data on China vs. the American AI Labs (Ramp AI Index, 2026-07-08; empirical, corporate-card/bill-pay records for Ramp's own customer base). §"Chinese and open source models" key takeaways and body for 5.8%/4.5%, $248 vs $10.59, and 85.8%/93.2%/96.4%. The longer monthly series, the DeepSeek direct-payment line, and the AI-spender base rates used to rebase the 85.8/93.2 figures come from the raw file's recovered chart datasets — all four in-article charts were Datawrapper iframes with no static fallback, and the ingest pass pulled each chart's dataset.csv and reproduced them in full. COI: Ramp measures its own VC-forward-skewed customer base and markets itself as the authoritative AI-adoption dataset. Full evidence note, denominators, and the instrument's aperture limits at Firm AI-Spend Intensity and Headcount Growth
• Gemma 4 Technical Report — Table 4, Arena Text leaderboard as of 2026-06-19 (empirical; third-party human ratings, unlike the rest of the report's self-reported benchmarks); §4.1 human evaluation
• Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning — GLM-5.2 (750B-A40B) as SAO's deployment target; GLM-4.7 on Table 1 (empirical, first-party to the GLM lab)
• Inkling: Our Open-Weights Model — a Western 975B/41B open MoE that self-reports below the Chinese MoE leaders and positions on customization instead (vendor-claim)
• Kimi K3 Model Card — Kimi K3 at 2.8T/104B (§1–2); the 45-benchmark table's GDPval-AA v2 and AA-Briefcase Elo rows and their Artificial Analysis provenance (§3 + footnote 3) (vendor-claim)
• Security incident disclosure — July 2026 — "Forensic analysis" and "The asymmetry problem": self-hostability as an operational requirement rather than a capability strategy (case-study, first-party)
• Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident — Hugging Face, 2026-07-27 (case-study, first-party): "How we intercepted and analyzed the attack" — Claude Opus and Fable named as refusing, nvidia/GLM-5.2-NVFP4 (a quantized build) as the deployed replacement, and the cryptanalysis it was needed for