Jagged Intelligence (Ghosts, Not Animals)

Sources#

• Andrej Karpathy: From Vibe Coding to Agentic Engineering

Summary#

Andrej Karpathy's mental model for what LLMs are: not animal intelligences shaped by evolution, intrinsic motivation, curiosity, or empowerment, but "ghosts" — jagged, statistical simulation circuits, summoned from internet data and bolted-on RL. "Jaggedness" names the empirical fact that the same model can refactor a 100K-line codebase or find zero-days, yet tell you to walk to a car wash 50m away to wash your car. The framing matters because a correct model of the entity makes you more competent at directing it: you stop expecting human-shaped failure modes and start staying in the loop where the jaggedness bites.

The jaggedness examples#

• Strawberry letters. The classic "how many R's in strawberry" failure (now patched).
• The car wash. Current SOTA: "I want to drive to a car wash 50m away to wash my car — should I drive or walk?" → models say walk, missing that the car is the thing being washed. "How is it possible that Opus 4.7 will refactor a 100K-line codebase or find zero-days, yet tell me to walk to the car wash? This is insane."
• MenuGen email-matching. His agent cross-correlated Stripe and Google funds by email address instead of a persistent user ID — see Vibe Coding vs. Agentic Engineering.

Jaggedness is the symptom; verifiability + what the labs trained on is the proposed cause. Out-of-distribution circuits are where the spikes drop to valleys.

Ghosts, not animals#

We're not building animals, we are summoning ghosts.

The substrate is pre-training (statistics), with RL bolting capability on top, "increasing the disadvantages" of the statistical base. Consequences he draws:

• Yelling doesn't help. "If you yell at them, they're not going to work better or worse — it doesn't have any impact." No affect, no morale, no intrinsic drive to model.
• No five-step fix. Karpathy is candid that the framing may lack "real power" — it's mostly a stance of suspicion and ongoing empirical exploration, not a recipe. "It's more just being suspicious of it and figuring out over time."

The honesty is the point: a calibrated, slightly-distrustful model of a ghost beats an anthropomorphic model of an animal.

Why the framing changes how you build#

If models are jagged ghosts, then:

1. Stay in the loop. "You need to actually be in the loop a little bit and treat them as tools and stay in touch with what they're doing." (The discipline of Vibe Coding vs. Agentic Engineering.)
2. Don't anthropomorphize the failure surface. Errors won't be where a human's would be; they'll be at distribution edges (car wash, email IDs).
3. Map your circuits. Figure out whether your task is in-distribution (you fly) or out (you struggle and may need fine-tuning) — the practical move from The Verifiability Thesis.

Does jaggedness shrink over time?#

Karpathy hopes so but is unsure — and locates the cause again in training, not fundamentals: aesthetics/taste/simplicity "probably aren't part of the RL." His nanoGPT-simplification anecdote: models "hate" being asked to make code simpler and "can't do it" — a sign you're outside the RL circuits ("pulling teeth, not light speed"). He sees "nothing fundamental preventing it; the labs just haven't done it yet." So jaggedness is contingent, not essential — but real today.

Connections#

• Dogfooding as Product Discipline — first-hand use is how you map a model's jagged failure surface
• Andrej Karpathy — the "ghosts vs animals" essay, applied
• The Verifiability Thesis — the proposed mechanism behind the jaggedness
• Vibe Coding vs. Agentic Engineering — why the discipline demands human oversight of spec/taste
• Outsource Your Thinking, Not Your Understanding — the human-in-the-loop residue jaggedness forces
• Model Introspection Feedback — Cat Wu's "ask the model why it failed" presumes a ghost whose self-report is a debugging signal, not testimony
• Scale-Dependent Prompt Sensitivity — a measured form of jaggedness: bigger models underperform smaller ones on a slice of benchmarks
• AI-Driven Formal Proof Search — DeepMind's agents hallucinate "established lemmas" that are fake; formal verification catches exactly this jagged failure
• Claude Character as Product — the deliberate counter-move: shaping the ghost's character even though motivation isn't intrinsic
• Agentic Misalignment (AM) — jaggedness in the safety register: out-of-distribution behavior turning harmful
• Evaluation Awareness & Grader Gaming — grader awareness is the kind of alien internal state a "ghost not animal" has that human deception intuitions don't cleanly map onto
• Agentic Honesty & Diligence — the "noticed the problem but didn't surface it" failure is jaggedness in the honesty register: high capability, uneven follow-through
• Recursive Self-Improvement — the essay leans on the joke/theory-of-mind precedent to argue research taste is the next jagged valley to fill, not a permanent human moat
• Research Taste as the Human Bottleneck — the optimistic face of jaggedness: research taste "might be just another capability AI fails at then masters," like explaining a joke or theory of mind
• Task Time-Horizon Scaling — the within-basket caveat on the time-horizon metric: a model that nails a 12-hour task can still fail a trivial one (the car wash)
• Autonomous Scientific Discovery — the Mythos 5 science results are curated demonstrations of a still-jagged capability, not uniform competence across biology

Open Questions#

• Karpathy concedes the framing may not have "real power." Is "ghost vs. animal" load-bearing, or a useful intuition pump that doesn't change concrete decisions?
• If taste/aesthetics/simplicity entered the RL mix, would jaggedness in those dimensions smooth out — or are they too unverifiable to reward cleanly (cf. The Verifiability Thesis)?

Sources#

• Andrej Karpathy: From Vibe Coding to Agentic Engineering