Noam Brown

Sources#

• Really Big Test-Time Compute in AI Changes Benchmarks, Safety and Research with OpenAI's Noam Brown

Summary#

Noam Brown is a research scientist at OpenAI and one of the researchers credited with pioneering inference-time (test-time) compute scaling — the reasoning paradigm behind the GPT-5-series "thinking" models. Before frontier LLMs he built superhuman poker agents (the Libratus / Pluribus line of work), and he still uses building a poker solver from scratch as his personal capability eval. In this corpus he is the sole author-subject of the June 2026 No Priors interview about his essay Implications of Large-Scale Test-Time Compute.

What he argues (in this corpus)#

Brown's essay hangs on one root claim — capability is now a function of inference budget — with several consequences he traces:

• The benchmark grid is broken. Single-number benchmark tables don't control for test-time compute, so a more efficient model (GPT-5.5) can look only marginally better than its predecessor while being a substantial jump. Fix: put cost/tokens/time on the x-axis. See Compute-Controlled Benchmarking.
• Safety evals are ill-defined at unbounded budgets. Preparedness frameworks / responsible scaling policies were built for the ChatGPT era and don't ask "at what budget do you evaluate?" — yet dangerous capability scales with dollars just like useful capability.
• No overnight intelligence explosion. Because peak capability requires large test-time-compute runs, time becomes the binding constraint; takeoff is gradual, not instantaneous. See Intelligence Explosion Dynamics.
• Research taste is the residual human role. Models optimize his poker algorithms 10–100× but cannot yet invent a better one; they're "a very good complement to researchers," not a replacement — though he expects an inflection point here like the ones in coding and math. See Research Taste as the Human Bottleneck.
• A latent-capability overhang exists. Nobody has explored what $100K of compute into a released model could do; the Erdős unit distance conjecture was disprovable from a public model before OpenAI announced it. See Latent Capability Overhang.

Brown's essay is practitioner-opinion — arguments and anecdotes from one lab. In July 2026 the UK AI Security Institute published the first independent, empirical corroboration of its core cluster, measuring capability curves over token budget across several benchmarks. The AISI cyber result Brown cited (models "still improving at 100M tokens") is that institute's own work, now published in full. His thesis is no longer sourced to a single person.

The poker eval (his signature instrument)#

Brown makes poker solvers because there's little open-source code for them, plenty of published theory, and "a lot of small gotchas" he's already worked through — so he can see exactly where a model fails. The progression he reports doubles as a capability timeline: early models "could not basically do anything"; GPT-5.2 could build a river solver with steering (and "felt like a grad student"), but "gaslit" him — famously insisting that folding a $100 pot loses $92, "it's close to 100, it's fine"; GPT-5.5 does much of it zero-shot. His forecast: within a year a model does "basically my entire PhD thesis in one go." He also uses models day-to-day for high-stakes non-code decisions (tax, real-estate paperwork), trusting their output "arguably more than… an expert human."

Connections#

• Large-Scale Test-Time Compute — his central thesis; he is the author of the essay this cluster is built on
• Compute-Controlled Benchmarking — his benchmark-grid critique and the "put compute on the x-axis" prescription
• Latent Capability Overhang — his observation that released models hold unextracted capability
• Research Taste as the Human Bottleneck — his practitioner reading: taste is the residue models fail at "for a time, then get good at"
• Intelligence Explosion Dynamics — his argument that test-time-compute dependence makes time the takeoff bottleneck
• OpenAI — his employer; the lab whose internal-model Erdős disproof and product-culture choices he reports
• UK AI Security Institute — the government evaluator whose July 2026 study independently, empirically corroborates his test-time-compute thesis

Sources#

• Really Big Test-Time Compute in AI Changes Benchmarks, Safety and Research with OpenAI's Noam Brown — No Priors interview with Sarah Guo (2026-06-26); Brown on his essay Implications of Large-Scale Test-Time Compute (practitioner-opinion)