Entities, in order.

Open questions 43 open

• AlphaProof Nexus
  • The framework's reach is gated by [[lean]]'s mathlib maturity. What's the path to domains needing new theory rather than subgoal decomposition?
  • AlphaProof adds little as a soloist but helps as a tool. As the prover LLM strengthens, does the AlphaProof tool become redundant entirely?
• Anthropic Institute
  • How does the Institute's policy posture (favoring an *option* to pause) interact with Anthropic's commercial incentive to ship frontier models? The essay acknowledges the competitive/geopolitical pressure but doesn't resolve it.
  • What concrete verification mechanisms will the Institute prototype, and on what timeline relative to the RSI trend it warns about?
• Campfire
  • Campfire claims its AI edge comes from "our own foundation model." For an ERP, what does a custom foundation model actually buy over fine-tuning a frontier model — and is it durable as frontier models improve (cf. [[harness-shrinkage-as-models-improve]])?
  • "Never had anyone outgrow Campfire" — does that hold as customers reach true enterprise scale where NetSuite's breadth historically mattered?
• Claude Design
  • Did the "any design tool via MCP" integration actually ship on the stated timeline? (Forward claim from May 2026.)
  • How does Claude Design's eval discipline work for visual/aesthetic output, where there's no compiler or test? (Same open question as [[cowork]] for non-code artifacts; relates to [[wiki/derived/evals-for-taste-and-character|character/taste evals]].)
• Claude Fable 5
  • **Why was access suspended after launch?** The source banner gives no reason (capacity? a safety finding? the UK-AISI jailbreak progress noted in [[capability-gated-model-fallback]]?). Not in source.
  • Exact benchmark numbers vs GPT-5.x / Gemini are image-only in the source; not transcribed.
  • How much of Fable's general-access experience is *actually* Fable vs Opus-4.8 fallback for security-research-adjacent users whose queries trip the conservative classifiers?
• Claude Mythos 5
  • **Suspension reason** — shared with Fable 5; not stated in source.
  • How does "somewhat stronger than Mythos Preview" square with Opus 4.8's card claiming Mythos Preview was the capability frontier? The frontier has moved; the magnitude isn't quantified here.
  • The bio trusted-access SKU is "Fable 5 with bio safeguards removed," not Mythos 5 — so "Mythos 5" strictly denotes the cyber-lifted variant. Whether these converge under one trusted-access umbrella is unstated.
• Claude Opus 4.7
  • Do Hakim's (2026) brevity-constraint findings on Opus 4.6 replicate on Opus 4.7, or does the literal-instruction-following change the elasticity? Specifically: does `<50 words` still yield +13.1pp on GSM8K?
  • Does Opus 4.7 still underperform as a planner in HotpotQA-style combo sweeps, or does improved instruction-following close the gap that AgentOpt (Hua et al., 2026) identified?
  • What is the real-world token-inflation multiplier on typical Claude Code sessions (1.0–1.35× is content-dependent — what's the distribution on code-heavy vs. prose-heavy inputs)?
  • How does xhigh compare to max on coding evals? The migration guidance says "start with high or xhigh" — is max ever worth it for coding?
  • What fraction of existing CLAUDE.md / system-prompt hedges become counterproductive under literal instruction following?
• Claude Opus 4.8
  • Public model ID and pricing: the card does not state them; presumably `claude-opus-4-8` at the Opus tier.
  • Does the grader-speculation trend continue to escalate in the next model, and at what point does it begin to affect outward behavior?
  • Why is 4.8 *less* robust to prompt injection than 4.7 despite broad alignment gains — a capability/robustness tradeoff, or an artifact of the eval surface?
• Cowork
  • How does Cowork's harness compare to [[claude-code]]'s? Both surface skills, MCP, sub-agents — but the failure modes for non-code output differ (no test suite, no compiler, no diff to review).
  • What's the eval discipline for Cowork-class outputs? Cat Wu says memory benefits a lot from evals; unclear how slide-deck quality is measured.
• Google DeepMind
  • DeepMind reports its bespoke systems being caught by simple loops. Does the lab's comparative advantage move from *systems* to *models + verifiers + benchmarks* (mathlib, Formal Conjectures)?
  • The paper opens AI-for-math; what's DeepMind's next target domain where a sound verifier exists?
• Hermes Agent
  • The container backend disabling dangerous-command checks is a defensible design but a meaningful security-model shift. What's the empirical track record? Have lockdown failures in popular images (Daytona, `nikolaik/python-nodejs`) caused incidents?
  • How do bounded memory files (~2,200 chars `MEMORY.md`) hold up over long-term use? Auto-consolidation is mentioned but not specified — what's the consolidation algorithm and how lossy is it?
  • Hermes's DM-pairing flow is a clean security primitive. Why hasn't this pattern been adopted by Claude Code or Cursor for shared/team deployments?
  • The split between `AGENTS.md` (project) and `SOUL.md` (personality) is explicit in Hermes but implicit in Claude Code's `CLAUDE.md`. Does the split materially improve outcomes, or is it a documentation choice without empirical backing?
  • Cron jobs in fresh sessions with no memory — how do teams structure the "context the agent needs" without it bloating every cron prompt? Is there a standard pattern?
• Lean
  • mathlib maturity gates the reachable frontier. Can AI formal proof search *grow* mathlib (formalize new theory) as a byproduct, expanding its own frontier?
  • Lean is a perfect verifier for math. Which other domains have a comparably sound automatic verifier (vs. only noisy ones like tests or LLM-judge councils)?
• METR
  • What new tasks will METR build to measure days- and weeks-long horizons once current baskets saturate?
  • METR also runs the [research showing developer self-estimates of AI uplift are overstated](https://arxiv.org/pdf/2507.09089) — how does it reconcile that skepticism with its own steep time-horizon curve?
• Mythos Model
  • Public release timeline: **answered** — Mythos Preview itself never shipped GA, but its descendants [[claude-fable-5|Fable 5]] / [[claude-mythos-5|Mythos 5]] reached general access in June 2026 (see *the descendants shipped* above). Both were suspended shortly after launch; whether and when they return is open.
  • Capability profile beyond cybersecurity: Mythos Preview focused on the safety story; other capability dimensions not well-documented externally.
  • Internal access controls: who at Anthropic actually uses Mythos for daily work, vs Opus 4.7? Boris implies infrequent (try-it use); not detailed.
• Symphony
  • The **500% landed-PRs claim** is hedged — no baseline definition, "on some teams" only. What does the distribution look like across teams? What happens to PR *quality* and revert rate at that throughput?
  • "Workspaces preserved across runs" is the opposite of typical CI ephemerality. At what point does state pollution from prior runs (stale `node_modules`, leftover branches, build artifacts) start hurting more than warm-cache helps?
  • Symphony doesn't write to the tracker — agents do. This means tracker policy is a *prompt* in `WORKFLOW.md`. How brittle is this in practice when Linear changes its API? How is consistent state-machine behavior enforced when agents have prompt-level discretion?
  • The spec was simplified by being implemented in 6 languages. What's the extension of this technique? Could `compiler-prompt.md` in this vault be similarly cross-fuzzed?
  • Symphony explicitly says agents can self-create tickets. What governance prevents runaway ticket-graph expansion? Is human triage of agent-created tickets the only check?