Research Taste as the Human Bottleneck

Sources#

• Jeff Dean: The 1% Rule for Building in AI
• Really Big Test-Time Compute in AI Changes Benchmarks, Safety and Research with OpenAI's Noam Brown
• Thread by @AndrewYNg
• When AI builds itself

Summary#

As AI absorbs the execution of AI development (AI Accelerating AI Development), the human role contracts toward a residue the essay When AI builds itself calls research taste and judgment: "choosing which problems matter, which results to trust, and when an approach is a dead end." This is the top rung of the autonomy ladder and the single capability that, if it stays human, keeps recursive self-improvement from closing the loop. Whether it stays human is the essay's load-bearing uncertainty.

The narrowing role#

The essay's clearest statement of the dynamic: "The human role is narrowing at each step in the AI development process." Two concrete narrowings:

• Writing → reviewing. "Once human- and AI-authored code quality reach parity, humans will stop writing code entirely, and shift to only reviewing it." (Parity is reported as roughly now; see AI Accelerating AI Development.) This is the harness-shrinkage story told from the human side.
• Running → choosing experiments. "Once Claude can run experiments, the question shifts towards 'Which of these experiments is worth running?'" The doing — writing the code, running the experiment, producing the result — "now costs almost nothing in human time, even if it still has costs in compute."

The residual human comparative advantage, "for now," is taste: deciding what's worth the compute. Put in the terms of Compute Allocator, the human becomes a compute allocator at the level of an entire research program rather than a single invocation.

Why it might not stay human — "just another capability"#

The essay refuses to treat research taste as a permanent human moat. Two arguments push against it:

1. Perspiration is automatable, and that's most of the work. Genius is "1% inspiration and 99% perspiration"; the 99% — scale it up, see what breaks, fix it, retry — is exactly what Claude excels at. Large-scale research progress "is mostly a function of tools and resources." (See The Bitter Lesson, Recursive Self-Improvement.)
2. Taste shows the same capability curve as everything else. Early evidence of improving research judgment — Claude beating the human next-step choice 51%→64% on hard detour moments (AI Accelerating AI Development) — suggests "research taste might be just another AI capability that AI systems fail at for a time, then get good at." The precedent: AI was once bad at, then good at, explaining why a joke is funny, demonstrating theory of mind, and solving linguistic riddles — qualitative skills assumed to be human-shaped. This is the optimistic face of Jagged Intelligence (Ghosts, Not Animals): today's valley is tomorrow's peak.

A frontier researcher's practitioner reading lands on the "just another capability" side. Noam Brown (OpenAI, practitioner-opinion) reports that models will optimize the algorithms from his poker research 10–100× but cannot yet invent a better one — "I can give it a lot of time and it's still not able to do it" — so today's models are "a very good complement to researchers," "not able to fully replace the whole research cycle." But he explicitly expects this to fall: "I wouldn't be surprised if at some point — same thing with coding, same thing with math — there's just this inflection point where suddenly it's actually good enough… I wouldn't be surprised if we encounter that point for research taste as well." This is the jagged valley argument stated by someone watching the valley from inside a lab.

The honest counter, in an Anthropic employee's words: "The comparative advantage of humans as of right now is still in seeing the bigger picture and thinking beyond the confines of the immediate task." The question is whether "for right now" is a stable equilibrium or a receding frontier.

The third answer: it isn't a capability at all#

Both arguments above — taste as a durable moat, taste as the next jagged valley — presuppose that taste is a capability, something a mind either has or acquires. Andrew Ng rejects the premise (June 2026, practitioner-opinion):

"Many people describe this human contribution as 'taste,' but I prefer to think of it as humans having a context advantage, since that gives us a clearer path to helping AI systems get better… So long as the human knows something the AI does not, human-in-the-loop is needed to inject that knowledge into the system."

Under this reading the residue is not a faculty but an information asymmetry — the human knows the users, the constraints, the history, the thing so obvious nobody wrote it down. That makes the human role falsifiable (does the asymmetry exist?), perishable (it closes as context transfers), and an engineering problem rather than a mystique. It also means the question this page is organized around — ceiling or valley? — may be malformed. See Context Advantage, Not Taste for the full treatment, including where the reframe fails: it explains the deployment asymmetry (I know my users) but not the generative one (The Abstraction Barrier, Transformative Creativity), where the missing thing is a concept, not a fact.

The two frames disagree about what would count as evidence. Returns to Expertise in Agentic Coding is the instrument that distinguishes them: if the expertise premium declines because models acquired judgment, taste was a capability; if it declines because context transfer improved, taste was an asymmetry all along.

The missing half: how you acquire it#

Every argument above is about whether taste is durable. None is about whether it is teachable — which matters, because if the residual human role is taste and taste is innate, the role doesn't scale to the people entering it. Jeff Dean (YC Startup School 2026, practitioner-opinion) is the corpus's first source to treat it as trainable and to name a protocol. He starts from the same position as Brown — asked what becomes scarce once agents write all the code, he answers "incredibly good taste in what you ask your agents to work on," and adds "I think models are not necessarily going to be that good at it" — then refuses to leave it mystical ("it's not like there's a measurable objective of taste").

Three mechanisms, in increasing order of how deliberate they are:

• Breadth of prior problems. The conventional answer: having worked on many things teaches you "what kinds of things might be just barely possible by cobbling together these previous approaches."
• Score your own forecasts. The one that is actually a protocol: "write down a bunch of things you think might be important in the next 12 months. Maybe you pick one of them to work on, but go back and evaluate in 12 months which of these other things actually seemed important, or which ones did other people in the world go out and create, and which ones did not seem to happen yet. That can give you a lot more samples for your own taste creation capability." The move is to manufacture training data for yourself — the counterfactual branches you didn't take get graded by the world, so one year of work yields N labeled examples instead of one.
• Attack a load-bearing assumption on purpose. His live example: 60 years of chip fabrication assume every chip is identical and no bit flips, while large-scale distributed systems assume the opposite and build reliability above unreliable parts — so what would you build out of transistors with 20 errors per day? He is explicit that most such experiments fail ("there are very good reasons that for the last 50 years we've done this thing this way"), which is the point: the yield is the sampling, not the hit rate.

Why this reframes the page's central question. If taste is a sample-efficiency problem — a small number of expensive, slowly-graded observations about what mattered — then it is neither a permanent moat nor an ordinary capability, and the reason models lag is legible: the training signal arrives on a 12-month delay and mostly as counterfactuals nobody records. That is a third position alongside Ng's information asymmetry and the ceiling/valley pair, and it is the one with an obvious attack — write the predictions down where a model can read the grade. Dean does not draw that implication; it follows from his own protocol.

The bottleneck either way#

Even if taste stays human, it becomes the binding constraint — the Amdahl's-law bottleneck of the whole pipeline. If humans can't review code as fast as Claude generates it, "human review will become the bottleneck to AI development." And if humans spend most of their time on the single-digit fraction of work that is direction-setting, each human steers vastly more work — so the quality and throughput of human judgment becomes the scarce resource, exactly as Verification as the New Bottleneck predicts for verification and the HBR accountability-redesign work predicts for oversight. The risk is that the human nominally decides but actually rubber-stamps (the "not close to substituting for senior researchers" judgment quietly eroding into a formality).

The human cost (the quieter thread)#

The essay includes unusually candid employee quotes about what the narrowing feels like — worth recording because they name a cost the productivity charts don't:

• The collapse of the gift economy of small favors: asking a colleague "can you help me get this script running?" "created a little debt, a little mutual awareness. [Claude is] faster, it creates zero debt, but each of these is a lost bid for human collaboration."
• The vertigo of contingent relevance: "On days where everything works well, I can't help but think nothing I do matters … But then there are days where everything breaks and I don't understand why and I realize I have no idea what I've been up to anymore."

Connections#

• Recursive Self-Improvement — whether taste stays human decides which of the three futures obtains; the essay's central uncertainty
• AI Accelerating AI Development — the evidence that execution is already absorbed, leaving taste as the residue
• AI R&D Autonomy Evaluation (AECI) — "not close to substituting for senior Research Scientists/Engineers" is the formal version of "taste is still human"
• Jagged Intelligence (Ghosts, Not Animals) — the joke/theory-of-mind precedent for "a capability AI fails at, then masters"; taste as the current valley
• Autonomous Scientific Discovery — Mythos 5's autonomous hypothesis-generation (~80% preferred over Opus-class) and "only high-level human input" genomics are concrete chips at the taste moat
• Verification as the New Bottleneck — if taste/review can't keep pace with generation, judgment becomes the binding constraint
• Harness Shrinkage as Models Improve — the same role-narrowing dynamic from the harness side; what's left after the model-facing harness dissolves
• Compute Allocator — taste exercised as allocation: deciding which experiments are worth the compute
• The Bitter Lesson — "research progress is mostly tools and resources" is the bitter lesson aimed at taste itself
• The Abstraction Barrier — DeepMind's theoretical case that taste/novel-concept-discovery is a real ceiling, not just the next capability to fall: AI may be bounded by human conceptual frameworks
• Transformative Creativity — Boden level-3 (creating new conceptual spaces) is taste exercised at the frontier of concept-space; the "Einstein test" version of the moat
• Multi-Agent Collective Intelligence — steering large superhuman-speed agent groups (whose output humans can't fully consume) is the form taste takes under the collective pathway
• Instrumental Convergence — a knowledge-seeking objective is an alternative framing of judgment as positive-sum information gain rather than scarce human taste
• Artificial Superintelligence (ASI) — taste/judgment is the candidate for what (if anything) stays human as systems cross from AGI into ASI
• Returns to Expertise in Agentic Coding — the labor-data instance of the same "is taste durable?" question: Anthropic frames a decrease in the returns to domain expertise over time as the signal that "models are starting to supply the essential judgment users currently bring" — taste-as-just-another-capability, made empirically trackable in usage
• Planning / Execution Division of Labor — the rubber-stamping risk made concrete: humans nominally own ~70% of planning decisions, but whether that control is real judgment or approval-by-default is exactly the "how do you measure rubber-stamping?" question
• Implementation Abundance Inverts Product Work — the product-work cousin: as implementation cheapens, curation/taste becomes the expensive step, the same residue named here for AI research
• Why AI Lags at Design — design taste as a currently-durable pocket of the human-as-reward-function; a concrete domain where the taste moat holds longer
• Role Averaging, Not Role Elimination — "taste-makers guide from inception" carries the same rubber-stamping risk: guidance is only real if it isn't approval-by-default
• Context Advantage, Not Taste — the third answer: Andrew Ng argues taste is not a capability but an information asymmetry, which makes it neither ceiling nor valley but a closable gap; dissolves this page's central question for product work while leaving it standing at the research frontier
• Unknowns as the Agentic Bottleneck — if the residue is context rather than taste, elicitation is the response; Thariq's blindspot passes and interviews are the protocol for pumping it across
• Noam Brown — a frontier researcher's practitioner reading: models optimize his algorithms 100× but can't invent a better one "for a time"; he expects an inflection point for research taste like the ones in coding and math
• Jeff Dean — the acquisition side the rest of this page skips: taste as a sample-efficiency problem, with a protocol (write 12-month forecasts, grade them against what the world actually built) that manufactures the labeled data the skill is starved of
• Researcher Uplift from Code Output — the quantitative shadow of this split: Kwa's model prices code/engineering uplift and treats research judgment as the unmodeled residual (assumes zero non-code uplift), matching Anthropic's "concentrated in engineering execution rather than research judgment"; a measured non-code uplift > 1 would be taste starting to fall

Open Questions#

• Is research taste a genuine ceiling (an architectural capability scaling can't reach) or the next jagged valley to fill? The essay calls this the decisive unknown. Contested premise: Ng argues the question presupposes taste is a capability at all.
• If taste is automatable, what — if anything — remains a durable human comparative advantage in AI development?
• How do you measure rubber-stamping? "Humans set direction" can be true on paper while real judgment quietly transfers to the model.

Sources#

• When AI builds itself — §"What might the future of work at Anthropic look like?" and §"What if we're wrong?"
• Thread by @AndrewYNg — Andrew Ng, The Batch (2026-06-30), practitioner-opinion: the "context advantage, not taste" reframing
• Really Big Test-Time Compute in AI Changes Benchmarks, Safety and Research with OpenAI's Noam Brown — Noam Brown (No Priors, 2026-06-26), practitioner-opinion: research taste as the residual human role; "not able to fully replace the whole research cycle," but expects an inflection point
• Jeff Dean: The 1% Rule for Building in AI — Jeff Dean, YC Startup School 2026 (2026-07-30, practitioner-opinion): §"How to Become an AI-Native Founder" and §"Question Your Biggest Assumptions" — taste as the scarce skill once agents write the code, "models are not necessarily going to be that good at it," and the three acquisition mechanisms including the graded-forecast protocol