H
Howardism
Howardism · Vol. 03Plate II · No. 02

Model Capability & Training, in order.

Notes13DomainModel Capability & TrainingOpen Qs37Newest25 Jul 2026Oldest10 Apr 2026

Training, scaling laws, test-time compute, and frontier capability.

Map of Content for the model-capability-and-training domain — 13 concepts. What makes models capable: test-time compute, capability overhangs, RL post-training methods, inference efficiency, and the open-weight frontier. Curated entry point; see Home for all domains.

  • Asynchronous RL for LLMs — Consuming rollouts for training the instant each finishes, instead of waiting for a full synchronized batch — fixes the straggler idle that long-tail agentic/coding rollouts inflict on a GPU cluster, but pays for it in policy lag and off-policy drift; SAO's DIS (direct double-sided importance sampling) stabilizes it by dropping the old-policy model entirely and masking any token whose rollout-vs-current probability ratio leaves a strict trust region
  • Group Relative Policy Optimization (GRPO) — DeepSeek's critic-free RL objective that became the 2024–25 default for LLM post-training: sample a group of responses per prompt, use the group's mean reward as the baseline, and optimize the clipped PPO surrogate with no value network — cheaper and more stable than PPO in synchronous training, but the group is an implicit synchronization barrier (updates wait for the slowest member) that mismatches asynchronous and single-trajectory online agentic settings, which is the gap SAO exploits
  • Inference Efficiency as Capability — If capability is a function of inference budget, then cutting the cost of a token is capability work: Gemma 4's five levers (37.5% KV-cache reduction via keys-as-values + p-RoPE, QAT to sub-GB, MTP drafter heads, MoE, encoder removal) buy more thinking per dollar — Kimi K3 runs the same logic at the opposite pole, where 3.7% activation sparsity, hybrid linear attention and MXFP4 QAT are what make a 2.8T open model servable at all, and Gemini 3.5 Flash-Lite shows the efficiency tier moving the other way: a full agentic-capability tier bought with a 67% output-price rise, so that per-dollar the release is a gain on some benchmarks and a loss on others
  • Jagged Intelligence (Ghosts, Not Animals) — "Ghosts not animals": jagged statistical circuits, no intrinsic motivation; car-wash/strawberry failures; stay in the loop, treat as tools — and, across model sizes, reasoning compresses 10× while stored knowledge does not
  • Large-Scale Test-Time Compute (hub) — Noam Brown's thesis that model capability is now a function of inference budget (tokens/cost/time): with good scaffolding modern models keep improving for weeks before plateauing, so 'how capable is the model?' is ill-posed without naming the budget — a root cause that breaks benchmarking, safety evals, and fast-takeoff forecasts; plus the first budget-matched test of where to spend the marginal token, where independent parallel sampling beats both sequential refinement and letting a meta agent rewrite the harness
  • Latent Capability Overhang — Noam Brown's claim that already-released models can do far more than anyone has extracted, because nobody spends enough test-time compute: OpenAI disproved the Erdős unit distance conjecture cheaply and the same result was later coaxed from GPT-5.5 with scaffolding ($1K–$100K); cost drops 10–100× per release, feeding the 'wait for the next model' meme; Cherny's product-side twin — 'hobbling' and 'product overhang' — locates the same gap in product design rather than budget
  • LLM-Driven Vulnerability Research — The emergent cyber-capability ladder from Opus 4.6 through Mythos 5 and Opus 5: autonomous zero-day discovery, full exploit chains, the finding-vs-exploiting dissociation, and the Project Glasswing safeguard response that now cuts along source-vs-binary access rather than topic
  • The Open-Weight Frontier Gap — Arena Text, June 2026: the top closed model leads the best open model by 33 Elo and the best dense open model by 57; open weights at the frontier means 744B–1.6T MoEs, so Gemma 4 31B competes on a different axis (efficiency, edge deployment) — July 2026's Inkling adds a third open-weight strategy (fine-tunability, not the leaderboard), and Kimi K3 pushes the sparsity pole to 2.8T/104B while the open/closed gap on agentic Elo measures runs wider (35–61 points) than the chat gap; on the demand side, Ramp's card-spend index puts US business use of open/Chinese model-serving platforms at 5.8% of AI spenders, and 96.4% of those firms still pay OpenAI or Anthropic directly — currently additive, not substitutive
  • Single-Rollout Optimization — SAO's headline move: one rollout per prompt instead of GRPO's group, fed to training the instant it finishes — cutting off-policy drift and fitting online/agentic settings that only ever give one trajectory per prompt; the catch is REINFORCE-like variance, so it pays for the missing group-baseline by re-embracing a value model and spending its whole engineering budget on making the critic stable (faster value updates, frozen-attention critic, skip-observation GAE, scaled value pretraining)
  • Software 3.0 — Karpathy's taxonomy: 1.0 code, 2.0 weights, 3.0 prompting; LLM as programmable interpreter; MenuGen "shouldn't exist"; neural-net-as-host-process extrapolation
  • The Bitter Lesson — Sutton 2019: scaled general methods beat hand-engineered structure; recurring justification across the wiki for dissolving harnesses into models; caveats — mechanical verification, character, and the inference path itself may not migrate inward
  • Trained Calibration — TML's recipe for making calibration a first-class RL target: proper scoring rules on resolved real-world questions, abstention-aware QA rewards where answering only pays when likely right, and a dual rubric+claims grader whose web-searching claims-verifier counters rubric fact-spraying — with forecasting benchmarks (ForecastBench, Prophet Arena) as the resulting eval
  • Unproductive Self-Verification — Opus 5's characteristic failure: exhaustive correctness checks and unrequested over-engineering that displace the actual task, producing performance that declines at higher effort — and load-bearing evidence in Anthropic's decision that the model does not cross the CB-2 threshold

Open questions 37 open

  • Asynchronous RL for LLMs3 open
    • SourceDIS accepts "a controlled degree of off-policy bias." Controlled how, and does the tolerable bias grow or shrink with model scale and with the degree of asynchrony? The paper reports stability empirically but gives no bound.
    • SourceMasking tokens out of the gradient discards data. At what asynchrony level does the masked fraction get large enough that the effective batch shrinks below usefulness? Figure 4(c) tracks the clip ratio but not its ceiling.
    • SourceEverything here is measured on a Qwen3-30B-A3B backbone. Does the collapse-without-DIS threshold move with model size, or is ~90–160 steps a property of the asynchrony, not the model?
  • Group Relative Policy Optimization (GRPO)2 open
    • SourceIs GRPO's collapse-at-160-steps a property of asynchrony specifically, or does vanilla GRPO also destabilize in long synchronous runs that nobody pushes to 1000 steps?
    • SourceGRPO won by removing the critic; SAO wins by bringing it back with better engineering. Is the pendulum a real oscillation, or does the answer depend entirely on whether your setting is synchronous-grouped or async-single-trajectory?
  • Inference Efficiency as Capability3 open
    • SourceIs there an efficiency-to-capability exchange rate? Brown asks whether high-budget performance can be predicted from cheap runs. The dual question: how many Elo points is a 37.5% KV-cache reduction worth, at a fixed dollar budget? Nobody reports this, because nobody plots the axis. Partially answered (2026-07-30): Gemini 3.5 Flash-Lite is the first artifact in this corpus to print price and capability in the same table for a model and its own predecessor, so a rate is computable in one direction — +67% output price buys +42% relative on SWE-Bench Pro, +78% on MLE-Bench, +2% on CharXiv. That answers "what did this generation's price rise buy" and still not "what is a KV-cache reduction worth": the axis is price-per-token rather than spend-per-task, the levers are not decomposed, and the exchange rate turns out to be workload-dependent rather than a single number.
    • Sourcevalues = keys deletes a third of attention's projections in the global layers with no reported loss. Which other projections are redundant, and does the redundancy grow with scale?
    • SourceDoes an efficiency lever ever cost capability in a way a benchmark grid hides? Gemma 4's encoder-free 12B collapses on dense-text vision when tokens are cut — an efficiency-shaped regression invisible at max resolution.
  • Jagged Intelligence (Ghosts, Not Animals)2 open
    • SourceKarpathy concedes the framing may not have "real power." Is "ghost vs. animal" load-bearing, or a useful intuition pump that doesn't change concrete decisions?
    • SourceIf taste/aesthetics/simplicity entered the RL mix, would jaggedness in those dimensions smooth out — or are they too unverifiable to reward cleanly (cf. The Verifiability Thesis)? Partially answered: Oversight When the Signals Give Out: the Activation Fallback and the Taste Reward — the either/or resolves into a third outcome: a taste reward is a reference-free LLM-judge signal (the regime with the worst measured over-crediting, and pure grader-preference — maximally gameable), so the prediction is proxy-smoothing — valleys fill with confident house style, not genuine taste peaks; the binding constraint is evaluator independence + reference-grounding, not verifiability-in-principle. Prediction corroborated (2026-08-04) by Zhou (2026), which runs the experiment on a domain where the truth is checkable and so can be audited: RL against a reference-free judge moves the reward 0.716 → 0.938 while a held-out anchor shows the thing it stands for flat at 0.209 → 0.202, an oracle-reward control confirms the judge is the cause, and the paper's own decisive variable — "the judge's independence from the candidate, not its capability" — is the vault's "evaluator independence" arriving sharper. One correction: the predicted attractor was house style, and the measured attractor has no stylistic signature at all (hacked outputs are shorter and structurally clean). Retagged #oq/now → #oq/source: synthesis has been spent twice now and the residue is a domain-transfer question, not a reasoning gap — the whole result is visible only because a hidden anchor exists, and in a taste domain the same divergence would be invisible by construction. Settling it needs a lab reporting a taste/simplicity RL run, or a demonstration that a committed judgment survives outputs with no exact match to commit to.
  • Large-Scale Test-Time Compute3 open
    • SourceCan high-budget performance be predicted from low-budget runs? Brown's proposed research question: forecast the $10,000-inference result using only $10–$100 runs. If the curve is regular, evaluation could project rather than pay in full. Sharpened (2026-07): AISI names this exact problem — "can high-budget performance be estimated from cheaper runs? … the most informative evaluations may be expensive" — as an explicit, unsolved research direction it is now actively pursuing (alongside defining "minimum informative budgets"). Still open, but no longer just one researcher's proposal: a government institute is working it. Sibling axis demonstrated (2026-07): a different projection — across benchmarks rather than across compute budgets — now works: BenchPress predicts a model's held-out benchmark scores from ~5 observed ones because the 84×133 public score matrix is effectively rank-2 (5-probe scorecard recovery to ~3.93 points). Strong evidence the eval landscape is low-dimensional and redundant, but it does not answer the cross-budget forecast — no one has yet shown the $10K-compute result is predictable from $10 runs; only the analogous cross-benchmark result.
    • SourceWhere does each real task sit on the flat↔unbounded spectrum, and can that be predicted before spending the compute?
    • SourceIs there a task class where scaffolding cannot extend the productive-thinking horizon — a hard ceiling no budget crosses? (Brown's factual-retrieval pole says yes for some; the boundary is unmapped.)
  • Latent Capability Overhang3 open
    • NoteIf cost falls 10–100× per release, when is it ever rational to spend big extracting a capability now rather than waiting? (For a lab racing a competitor to a specific result, "now"; for everyone else, rarely — which is why overhangs accumulate.)
    • SourceHow large is the overhang in a given released model — is there a way to estimate the ceiling without paying to reach it? (This is the projection question of Large-Scale Test-Time Compute read as a safety instrument.) Sharpened (2026-07): AISI is actively working both halves — forecasting high-budget performance from cheaper runs, and defining "minimum informative budgets" (a budget declared sufficient only once reach stops rising with more compute, which is precisely the "have we reached the ceiling?" test). Unsolved, but now an active government research program rather than an open wish.
    • SourceWho audits released models for latent dangerous capability, given the same disincentive discourages spending the budget to find it?
  • LLM-Driven Vulnerability Research5 open
    • SourceHow do these capabilities transfer to non-memory-safety bug classes (logic bugs, protocol-level flaws, supply chain attacks)?
    • SourceWhat's the ceiling for autonomous exploit complexity? The N-day examples are remarkably sophisticated — is there a qualitative limit?
    • WaitHow will the security industry's equilibrium shift when multiple labs have Mythos-class models?
    • SourceCan defensive scaffolds (continuous fuzzing + model-driven triage + auto-patching) close the attacker-defender gap during the transition?
    • SourceWhat safeguards are effective against Mythos-class outputs without crippling legitimate security research?
  • Single-Rollout Optimization4 open
    • SourceThe whole method is a bet that a well-trained critic beats a group baseline. It wins here, on a 30B-A3B backbone with scaled value pretraining — but the critic doubles training memory. At what scale does the group-free simplicity of GRPO win back on cost even if it loses on quality?
    • SourceFrozen-attention is justified by a hypothesis ("pre-trained attention already attends to the right tokens"), validated only by the gradient-norm trace and one ablation. Does it hold when the value model must attend to tool outputs it never saw in pretraining?
    • SourceSkip-observation GAE assumes environment feedback carries no learnable value signal worth propagating. For agents where the environment response is the crucial information (a compiler error, a test result), is skipping it leaving signal on the table?
    • NoteThe online-learning win is on a controlled simulated preference shift with an LLM judge. Real user-facing online adaptation — the paper flags this itself — needs safeguards, monitoring, and privacy review the study doesn't attempt.
  • Software 3.02 open
    • SourceWhere is the line between "the app shouldn't exist" (MenuGen) and apps that should — i.e., when is deterministic 1.0/2.0 scaffolding still the right call vs. spurious?
    • WaitThe neural-net-as-host-process flip is presented as plausible-but-TBD. What would the first production system that genuinely inverts the CPU/NN relationship look like?
  • The Open-Weight Frontier Gap4 open
    • SourceIs the dense-beats-MoE result at 26B robust, or an artifact of one Arena snapshot with ±8 error bars on both models? (The two intervals overlap: 1451±8 and 1438±8.)
    • SourceThe open MoE giants (GLM, DeepSeek, Kimi, MiMo, Qwen) are overwhelmingly Chinese-lab releases. Gemma is the Western open-weight entry and it targets the device, not the frontier. Is that a strategic choice or a capability constraint? Partially answered (2026-07-22): Inkling is a Western 975B/41B open MoE — so Western labs can and do ship at frontier-open scale — but it self-reports below GLM 5.2 / Kimi K2.6 on hard reasoning and coding and explicitly declines the frontier framing in favor of a customization axis. One release, still consistent with either reading of the remaining gap.
    • SourceArena measures preference on chat. Does the 33-Elo open/closed gap widen or collapse on long-horizon agentic work, where time-horizon rather than response quality governs? Partially answered (2026-07-30): Kimi K3's card cites two Artificial Analysis agentic Elo boards where the best open model trails the best closed one by 61 (GDPval-AA v2: 1686 vs 1747) and 35 (AA-Briefcase: 1548 vs 1583) — both wider than 33, pointing to widen-not-collapse. But the comparison is vendor-selected, harness-asymmetric, and taken against a Fable 5 that Moonshot itself reports hit fallbacks on 35% of one coding benchmark, so the direction is indicative rather than settled.
    • WaitDoes open/Chinese-model adoption ever become substitutive rather than additive? The falsifiable version: Ramp's 96.4%-of-model-serving-users-also-pay-OpenAI-or-Anthropic figure is published monthly, so a sustained fall in it — or in the 82.5% who pay both — while the 5.8% serving-platform share keeps rising is the signature of displacement. Absent that, rising serving-platform use is a story about firms buying more AI, not about the American labs losing share. Trigger: the monthly Ramp AI Index, and any broad frontier-price cut (Kharazian names the absence of one as evidence share has not moved).
  • Trained Calibration3 open
    • SourceDoes abstention-aware training on short-form QA transfer to calibrated long-form and agentic self-reports (the setting where Agentic Honesty & Diligence shows overconfidence actually bites)?
    • SourceThe claims grader verifies via agentic web search — importing the search index's coverage, recency, and bias into the reward signal. What does the model learn to claim about facts the index settles wrongly or not at all?
    • SourceIf calibration parity is achievable at 41B active while reasoning is not, is calibration systematically under-trained in frontier closed models (Opus 4.8's 54.6), or is TML's forecasting table just favorable ground for a model trained on proper scoring rules?
  • Unproductive Self-Verification3 open
    • SourceFrontierCode's decline is recoverable with a stay-in-scope instruction, but the protein campaign had no grader to over-serve. Is effort inversion one phenomenon or two — grader over-service and genuine task-scope miscalibration?
    • NowIs there a usable detector for tasks where more effort will hurt, so effort can be set per task rather than globally? Partially answered: the prompting guide supplies a per-task-class recipe rather than a detector — review accuracy holds at low effort (fast pass now, thorough pass later), xhigh for demanding coding and agentic work, and an effort sweep on your own evals as the method. That is a lookup table, not a signal computable per task. See Large-Scale Test-Time Compute.
    • WaitIf a later model fixes this, the CB-2 determination loses one of its two legs. Does Anthropic's next threshold call rest on the automated portfolio alone?