Fundamental Limits of ASI

Sources#

• From AGI to ASI

Summary#

A recurring corrective in the "From AGI to ASI" report: ASI is neither omniscient nor omnipotent. However capable, it is bound by hard limits — many of which are well understood. The subtle and important catch (Table 2 of the report): these limits are mostly worst-case negative results that are "vacuous" in practice, because approximate solutions and heuristics often achieve excellent performance far below the worst-case compute bound. So they rarely tell us whether a concrete capability (curing ageing, fusion, simulating a brain) is achievable.

The catalogue of hard limits (Table 2)#

• Fundamental physics — speed of light (information-propagation limit); Landauer's principle (minimum energy to erase a bit); Bremermann's limit (max computation speed per unit mass-energy); Bekenstein bound (max information in a finite region with finite energy).
• Real time — the physical world runs in real time. Experiments that can't be simulated with sufficient precision (weather, organisms, economies, societies) are gated by physical latency; large simulations take time (less with faster compute, but not zero). This is the seed of the embodied bottleneck.
• Physical manipulation — physical non-universality: not every logically-possible configuration of matter can be physically realized with finite space/energy (cf. Universal Constructor). Even realizable ones take time and cost energy/resources to build.
• Ignorance, observability & controllability — epistemic uncertainty and finite-precision measurement impose limits on predictability and control.
• Complexity theory — P vs NP vs PSPACE etc. Practical computability bounds apply to ASI too (though often worst-case, with good approximations available below them).
• Logic — Gödel's incompleteness and the Halting Problem: limits of what can be objectively answered or known. (These also bound provably-optimal self-modification — see Schmidhuber's Gödel machines under Intelligence Explosion Dynamics.)

Why the limits are hard to weaponize for prediction#

Predicting what ASI can't do is harder than it looks. Two routes both disappoint:

• Extrapolating from today becomes wildly uncertain very quickly.
• Theory mostly yields vacuous negatives: "ASI cannot play provably perfect chess" (exhaustive game-tree search is prohibitive) is true but irrelevant, because near-perfect heuristic play is cheap. What we'd need are theorems about problems that are hard and admit no good approximation — and current theory is far weaker here than on hard complexity bounds.

Worse, whether a good approximation exists (and how good, at what cost) may itself be computationally irreducible — provable only by running candidate programs from shortest upward. The report grounds this in Kolmogorov's structure function for lossy compression: most strings are incompressible (must be memorized); for compressible ones, how good a sub-minimal-length lossy compression can be cannot be predicted in advance, only discovered by running programs. Since compression ≡ prediction, Universal AI (and by extension ASI) inherits this unpredictability about its own approximate performance.

Consequence#

Capability prediction must be empirically-first, theory-complemented — scaling laws and benchmark stitching, not pure deduction (see Effective Compute Scaling and AGI-to-ASI Pathways's forecasting agenda). The hard limits set an outer fence; they don't draw the practical map.

Connections#

• Artificial Superintelligence (ASI) — the "neither omniscient nor omnipotent" claim this page substantiates
• Universal AI (AIXI) — AIXI formalizes the data-efficiency limit and inherits the lossy-compression unpredictability result
• The Abstraction Barrier — the embodied bottleneck is the "real-time / physical manipulation" limit applied to recursive improvement
• Intelligence Explosion Dynamics — physical/real-time limits are what bound an otherwise-explosive self-improvement loop
• Effective Compute Scaling — why forecasting is empirical-first: theory gives only vacuous negatives

Open Questions#

• Can we develop theory for "hard and inapproximable" problem classes — the only negatives with practical bite?
• How much slack sits between these fundamental limits and the practical ceiling of AGI/ASI systems?

Sources#

• From AGI to ASI — Section 3, Table 2 ("Fundamental limitations of ASI"); Section 6 ("Is it possible to predict what ASI can and cannot do?")