Jagged Intelligence (Ghosts, Not Animals)

Sources#

• Andrej Karpathy: From Vibe Coding to Agentic Engineering
• Claude Opus 5 System Card
• Gemma 4 Technical Report
• Verbalizable Representations Form a Global Workspace in Language Models

Summary#

Andrej Karpathy's mental model for what LLMs are: not animal intelligences shaped by evolution, intrinsic motivation, curiosity, or empowerment, but "ghosts" — jagged, statistical simulation circuits, summoned from internet data and bolted-on RL. "Jaggedness" names the empirical fact that the same model can refactor a 100K-line codebase or find zero-days, yet tell you to walk to a car wash 50m away to wash your car. The framing matters because a correct model of the entity makes you more competent at directing it: you stop expecting human-shaped failure modes and start staying in the loop where the jaggedness bites.

The jaggedness examples#

• Strawberry letters. The classic "how many R's in strawberry" failure (now patched).
• The car wash. Current SOTA: "I want to drive to a car wash 50m away to wash my car — should I drive or walk?" → models say walk, missing that the car is the thing being washed. "How is it possible that Opus 4.7 will refactor a 100K-line codebase or find zero-days, yet tell me to walk to the car wash? This is insane."
• MenuGen email-matching. His agent cross-correlated Stripe and Google funds by email address instead of a persistent user ID — see Vibe Coding vs. Agentic Engineering.

Jaggedness is the symptom; verifiability + what the labs trained on is the proposed cause. Out-of-distribution circuits are where the spikes drop to valleys.

Ghosts, not animals#

We're not building animals, we are summoning ghosts.

The substrate is pre-training (statistics), with RL bolting capability on top, "increasing the disadvantages" of the statistical base. Consequences he draws:

• Yelling doesn't help. "If you yell at them, they're not going to work better or worse — it doesn't have any impact." No affect, no morale, no intrinsic drive to model.
• No five-step fix. Karpathy is candid that the framing may lack "real power" — it's mostly a stance of suspicion and ongoing empirical exploration, not a recipe. "It's more just being suspicious of it and figuring out over time."

The honesty is the point: a calibrated, slightly-distrustful model of a ghost beats an anthropomorphic model of an animal.

Why the framing changes how you build#

If models are jagged ghosts, then:

1. Stay in the loop. "You need to actually be in the loop a little bit and treat them as tools and stay in touch with what they're doing." (The discipline of Vibe Coding vs. Agentic Engineering.)
2. Don't anthropomorphize the failure surface. Errors won't be where a human's would be; they'll be at distribution edges (car wash, email IDs).
3. Map your circuits. Figure out whether your task is in-distribution (you fly) or out (you struggle and may need fine-tuning) — the practical move from The Verifiability Thesis.

Jaggedness in the compression direction (Gemma 4, July 2026)#

Karpathy's examples are jaggedness across tasks at fixed model. Gemma 4 supplies a measured instance of jaggedness across model sizes, and it has a legible shape.

DeepMind claims its 2.3B E2B "roughly matches Gemma 3 27B performance with 10× less parameters." Reading Table 5 against that claim, the match is jagged rather than flat:

	E2B (2.3B)	Gemma 3 27B
AIME 2026	37.5	20.8
Codeforces Elo	633	110
LiveCodeBench v6	44.0	29.1
GPQA Diamond	43.4	42.4
MMLU Pro	60.0	67.6
MMMLU	67.4	70.7
τ²-airline	31.0	39.0

The pattern: reasoning compresses, stored knowledge does not. A 10× smaller model with a thinking mode wins by wide margins wherever the task rewards search over a small state space, and loses on the two broad-knowledge benchmarks, where the answer has to be in the weights. (Both comparisons are confounded by thinking mode — see Compute-Controlled Benchmarking — which if anything strengthens the reading: the reasoning wins are partly bought with inference, while nothing bought back the missing facts.)

The valleys get steeper below E2B's reasoning strengths. On GraphWalks the E2B scores 4.1 F1 against Gemma 3 27B's 32.8 — not a shortfall but a collapse. On Humanity's Last Exam, 31B manages 19.5, the 12B 5.2, and the two small models are not reported at all. Jaggedness at small scale is less a spiky frontier than a narrow ridge.

This is the same phenomenon Scale-Dependent Prompt Sensitivity measures from the other end (large models underperforming small ones on 7.7% of problems), and it suggests the jagged surface is not merely about which tasks but about which resource — parameters and inference budget buy different capabilities and are not fungible.

Does jaggedness shrink over time?#

Karpathy hopes so but is unsure — and locates the cause again in training, not fundamentals: aesthetics/taste/simplicity "probably aren't part of the RL." His nanoGPT-simplification anecdote: models "hate" being asked to make code simpler and "can't do it" — a sign you're outside the RL circuits ("pulling teeth, not light speed"). He sees "nothing fundamental preventing it; the labs just haven't done it yet." So jaggedness is contingent, not essential — but real today.

Connections#

• The Tragedy of the Cognitive Commons — Dell'Acqua's consultants gaining inside the capability frontier and losing outside it, unable to tell which side they were on, is the jagged edge restated as a validation requirement

• Task Saturation: Broad but Shallow AI Diffusion — jaggedness measured at economy scale: AI reaches 68% of occupations but a median 21% of their tasks, and the extensive margin is gated by physicality rather than difficulty

• Unproductive Self-Verification — jaggedness at its widest: IMO gold at 42/42 and a 24-hour autonomous design campaign that shipped nothing, in the same model

• Automatic vs. Flexible Cognition in LLMs — one edge of the jagged frontier gets a mechanism: tasks that survive J-space ablation (classification, span extraction, fluent continuation) versus those that collapse (multi-hop reasoning, translation, analogy, summarization) — 'automatic' vs. workspace-dependent, measurable by ablation rather than inferred from benchmarks

• Why AI Lags at Design — design as a current valley of the jagged frontier (a thing AI fails at until it doesn't)

• Dogfooding as Product Discipline — first-hand use is how you map a model's jagged failure surface

• Andrej Karpathy — the "ghosts vs animals" essay, applied

• The Verifiability Thesis — the proposed mechanism behind the jaggedness

• Vibe Coding vs. Agentic Engineering — why the discipline demands human oversight of spec/taste

• Outsource Your Thinking, Not Your Understanding — the human-in-the-loop residue jaggedness forces

• Model Introspection Feedback — Cat Wu's "ask the model why it failed" presumes a ghost whose self-report is a debugging signal, not testimony

• Scale-Dependent Prompt Sensitivity — a measured form of jaggedness: bigger models underperform smaller ones on a slice of benchmarks

• AI-Driven Formal Proof Search — DeepMind's agents hallucinate "established lemmas" that are fake; formal verification catches exactly this jagged failure

• Claude Character as Product — the deliberate counter-move: shaping the ghost's character even though motivation isn't intrinsic

• Agentic Misalignment (AM) — jaggedness in the safety register: out-of-distribution behavior turning harmful

• Evaluation Awareness & Grader Gaming — grader awareness is the kind of alien internal state a "ghost not animal" has that human deception intuitions don't cleanly map onto

• Agentic Honesty & Diligence — the "noticed the problem but didn't surface it" failure is jaggedness in the honesty register: high capability, uneven follow-through

• Recursive Self-Improvement — the essay leans on the joke/theory-of-mind precedent to argue research taste is the next jagged valley to fill, not a permanent human moat

• Research Taste as the Human Bottleneck — the optimistic face of jaggedness: research taste "might be just another capability AI fails at then masters," like explaining a joke or theory of mind

• Task Time-Horizon Scaling — the within-basket caveat on the time-horizon metric: a model that nails a 12-hour task can still fail a trivial one (the car wash)

• Autonomous Scientific Discovery — the Mythos 5 science results are curated demonstrations of a still-jagged capability, not uniform competence across biology

• Artificial Superintelligence (ASI) — Remark III of the DeepMind report: even if the Legg–Hutter score is smooth in compute, concrete capability profiles are jagged vs. human level

• Transformative Creativity — "creativity (like research taste) might be just another capability AI fails at then masters" mirrors the joke/theory-of-mind precedent

• Loop Engineering — "stay in the loop, treat them as tools" is the cure for the cognitive surrender failure mode of unattended loops; a self-prompting loop running on a jagged ghost makes the car-wash failures land unsupervised

• Returns to Expertise in Agentic Coding — staying-in-the-loop pays measurable dividends: in Anthropic's 400K-session data, experts recover from the agent's spiky failures (verified success 4%→15% among troubled sessions) while novices abandon them (19% vs 5–7%)

• Context Advantage, Not Taste — Andrew Ng displaces the framing this page supplies for taste: "a capability AI fails at then masters" presupposes taste is a capability, where he reads it as an information asymmetry

• The Open-Weight Frontier Gap — an aggregate Arena Elo averages the ridge flat; the small Gemmas' reasoning wins and knowledge losses both vanish into one number

• Compute-Controlled Benchmarking — the compression comparison is confounded by thinking mode, which sharpens rather than weakens the reasoning-vs-knowledge reading

• Gemma 4 — the measured instance: reasoning compresses 10×, stored knowledge doesn't

• Reference-Free Judge Over-Crediting — what a taste reward would be made of, measured on a task where the truth can be checked: optimizing against a reference-free judge moves the reward a long way (0.716 → 0.938) and the underlying capability not at all (0.209 → 0.202). The transferable part for this page is the bound — the gap is at most 1 − accuracy, so the regimes with the most room to smooth falsely are exactly the valleys, and the high-accuracy peaks show no exploitable gap (TruthfulQA at 0.893 base, gap −0.041). Jaggedness is what makes a proxy reward dangerous, not what it fixes

• Oversight When the Signals Give Out: the Activation Fallback and the Taste Reward — prices the taste-in-the-RL-mix question: with today's judge stack the predicted outcome is proxy-smoothing (house style fills the valleys), not genuine taste peaks

Open Questions#

• Karpathy concedes the framing may not have "real power." Is "ghost vs. animal" load-bearing, or a useful intuition pump that doesn't change concrete decisions?
• If taste/aesthetics/simplicity entered the RL mix, would jaggedness in those dimensions smooth out — or are they too unverifiable to reward cleanly (cf. The Verifiability Thesis)? Partially answered: Oversight When the Signals Give Out: the Activation Fallback and the Taste Reward — the either/or resolves into a third outcome: a taste reward is a reference-free LLM-judge signal (the regime with the worst measured over-crediting, and pure grader-preference — maximally gameable), so the prediction is proxy-smoothing — valleys fill with confident house style, not genuine taste peaks; the binding constraint is evaluator independence + reference-grounding, not verifiability-in-principle. Prediction corroborated (2026-08-04) by Zhou (2026), which runs the experiment on a domain where the truth is checkable and so can be audited: RL against a reference-free judge moves the reward 0.716 → 0.938 while a held-out anchor shows the thing it stands for flat at 0.209 → 0.202, an oracle-reward control confirms the judge is the cause, and the paper's own decisive variable — "the judge's independence from the candidate, not its capability" — is the vault's "evaluator independence" arriving sharper. One correction: the predicted attractor was house style, and the measured attractor has no stylistic signature at all (hacked outputs are shorter and structurally clean). Retagged #oq/now → #oq/source: synthesis has been spent twice now and the residue is a domain-transfer question, not a reasoning gap — the whole result is visible only because a hidden anchor exists, and in a taste domain the same divergence would be invisible by construction. Settling it needs a lab reporting a taste/simplicity RL run, or a demonstration that a committed judgment survives outputs with no exact match to commit to.

Sources#

• Andrej Karpathy: From Vibe Coding to Agentic Engineering
• Gemma 4 Technical Report — Table 5 (E2B vs Gemma 3 27B), Table 9 (GraphWalks collapse at 2.3B) (empirical)
• Verbalizable Representations Form a Global Workspace in Language Models — the selectivity result gives one edge of the jagged frontier a mechanism: workspace-independent ('automatic') tasks survive J-space ablation; workspace-dependent ones collapse