The Global Workspace in Language Models (J-space)

Sources#

• A Review of Anthropic's Global Workspace Paper
• jspace-replication: Independent Replication of Anthropic's Global-Workspace Paper on Small Open Models
• Verbalizable Representations Form a Global Workspace in Language Models

Summary#

Using the Jacobian Lens (J-lens), Anthropic's interpretability team searched for representations that are verbalizable — poised to be spoken about if the model were asked — and discovered, "rather surprisingly," that this set does far more than support speech. It behaves like a global workspace: a small, privileged, broadcast subset of the model's representations, sitting atop a much larger volume of automatic processing that the model cannot report on or flexibly reason with.

The J-space is the set of points expressible as a sparse non-negative combination of J-lens vectors (typically $k \le 25$). It is small: across workspace layers it never carries more than 10% of activation variance, and a concept's J-space component holds a median 6–7% of that concept's representational variance. And yet that sliver is where the causation lives.

The five functional properties#

Global workspace theory (Baars, Dehaene) says conscious access is reportable, top-down controllable, the medium of deliberate reasoning, flexibly generalizable, and selective. The paper defines an LLM analog of each and tests it.

1. Verbal report. Ask Sonnet 4.5 to think of a sport and read the lens at the colon before it answers: Soccer sits at the top and the model says "Soccer." Swap the Soccer coordinate for Rugby — a word not in its top-10 outputs — and it reports "Rugby." Systematically, swaps drive the target into the top-5 on 88% of trials. Injecting a J-lens vector on the user turn makes the model report detecting that concept when later asked to introspect, without making it blurt the word out earlier — the representation is verbalizable, not an unconditional impulse to speak.

The J-space is privileged for report. Decompose a concept vector into its J-space component (~6–7% of variance) and its non-J-space remainder (~93%): swapping along the J-space component succeeds on 59% of trials, approaching the 88% of pure J-lens vectors; swapping along the non-J-space component succeeds on 5%. Clamp the J-space coordinates so the concept cannot re-enter, and even that 5% falls to zero — the residual effect was routing through the workspace.

2. Directed modulation. Told to "concentrate on citrus fruits" while copying an unrelated sentence, orange is the top lens token across a band of layers — plus, at intermediate layers, tokens naming the act (imagine, thinking, focused). Told to mentally evaluate 3²−2 while copying, the readout runs arithmetic → nine → seven. None of this appears in the output. Modulation is also implicit: the same stimulus tokens load a property's label (past, adjective) into the J-space only when the question asks the model to name the property — even though the property is represented and used under both questions.

The control is imperfect in a familiar way: an "ignore X" instruction leaves X in the workspace well above a zero baseline — the white-bear effect, reproduced in a transformer.

3. Internal reasoning. J-lens vectors carry load-bearing unspoken intermediates:

• "The number of legs on the animal that spins webs is" — spider is in the lens though it appears in neither prompt nor output. Swap spider → ant and "8" becomes "6."
• Completing a rhyming couplet, the lens shows the planned rhyme fight before the line is written. Swap it for light and the model's earlier word choices change ("coming" → "morning"), ending on "morning light." The plan causally constrains the path to it. (Contested 2026-07-30: poetry planning failed to replicate on Qwen 3.6 27B — see External replication below. Nanda attributes the failure to experimenter error or the weaker model and does not adjudicate. A second audit deflates the class rather than the instance: once lens hits the model was about to say anyway are excluded, covert content survives mostly for context registers, not content plans — see A second independent audit below.)
• Asked in Chinese for the antonym of 小, the lens carries the English big — swap for long and the Chinese output changes 大 → 长. The model thinks in English and explicitly represents which language it must translate into.
• Two-hop swaps succeed on 54% (Haiku 4.5) / 70% (Sonnet 4.5) / 70% (Opus 4.5) of trials. The intermediate swap takes effect ~17% earlier in depth than the answer swap — ruling out the confound that the intermediate vector merely smuggles in the answer. (Contested 2026-07-30: on Qwen 3.6 27B answer-token swapping strictly dominated intermediate swapping, the reverse of the paper's Figure 15 ordering — though Nanda judges the replication dataset likely too easy. See External replication below. The missing control — a direct final-token substitution baseline — now exists in a third-party audit, results not yet published.)

Again privileged: probe-based intermediates flip the answer 61% via their J-space component vs. 28% via the non-J-space remainder — and that 28% collapses to 6% when the J-space is clamped.

4. Flexible generalization (broadcast). Swap France → China once, at every position, and every downstream function reads the new argument correctly: the capital, the language, the continent. Across 16 functions and 192 trials, 76 succeed (101 at double strength). Failures concentrate where the source concept's workspace loading was low — country names load strongly and swap reliably; number words load weakly and swap poorly.

5. Selectivity. The J-space is required for report and flexible inference and not for automatic processing — see Automatic vs. Flexible Cognition in LLMs, the sharpest result in the paper.

The three structural signatures#

A layer band. The J-space carries workspace-like content only between roughly L38 and L92 (of 100, reindexed). Before it: essentially no meaningful content (near-zero excess kurtosis, near-null autocorrelation, collapsed effective dimensionality). After it: a "motor" regime where the readout aligns with the imminent output token. Multiple independent lens statistics converge on the same boundaries. (Qualified 2026-07-30: the CKA band structure reproduces on Qwen 3.6 27B but is "notably less clean" — 2–3 overlapping bands out of 4–5 total rather than a crisp partition.)

Ignition at the onset. Feed the model an input embedding that is a weighted blend of two country names and sweep the mixture. In early layers the activation tracks the blend proportionally; starting at ~L38 it snaps to one endpoint or the other, switching sharply at a threshold — with bimodal outcomes at maximal ambiguity, especially in the J-space. This is the closest thing to GWT's all-or-none ignition, and it is measured without the J-lens (a plain projection-share measure), which is what makes the workspace-onset layer more than a lens artifact.

Limited capacity, and a broadcast hub.

• Occupancy plateaus around 25 J-lens vectors at a time. On a list of unrelated words only ~6 of those read so far are present at any comma (~1–2 at a single layer); on a list of related words nearly the whole 80-word category is present within a few items — including words not yet read. The model is holding the category, not recalling the list. Switch categories and the old items are evicted within a few words — it is the new category's arrival, not elapsed tokens, that clears the workspace.
• Broadcast across depth: MLP blocks amplify J-lens vectors ~10× relative to random directions (neuron output directions: ~1×), and the effect is monotone in how J-space-aligned an SAE feature is.
• Broadcast across tokens: a top-1% set of attention heads selectively relays J-space content — cleanly separated from broadcast heads for rotated-J controls, SAE strata, and MLP rows. Ablating them drops J-lens recall@25 to 0.67 (control: 0.86) while changing the model's top-1 next token at only 5% of positions (control: 2%) — they act on the workspace, not the output. Ablating them also collapses injected-thought reporting (0.54 → 0.09).

What the authors do not claim#

They explicitly refuse the strong version. Transformers have no separable specialist processors, no recurrence within a forward pass (the broadcast they document runs across depth, not through recurrent loops), and it is unclear whether workspace entry involves the sharp competitive ignition the brain shows. The claim is that the J-space achieves many of the functional properties of the global workspace while sharing only some of its architectural ones. On phenomenal consciousness they take no position — see Access-Consciousness Indicators in AI.

External review and independent replication (Nanda et al., 2026-07-06)#

Anthropic commissioned a public review from Neel Nanda, published the same day as the paper. It carries two kinds of content that should be weighted differently: review judgments (expert opinion, no measurement) and an independent replication on Qwen 3.6 27B with Camila Blank and Agam Bhatia (measured, but small-n and unadjudicated).

The judgment. Nanda finds the core scientific claim — that a cognitive space exists holding intermediate variables across a forward pass — compelling: "even if I quibbled over many details, there's enough hard-to-fake evidence that clearly something important is going on." He is markedly less confident in the fine-grained structural properties (the Section 4 material this page's "three structural signatures" is drawn from), where alternative hypotheses may exist and results may not generalize across models. He declines the philosophical claim entirely → Access-Consciousness Indicators in AI.

Which results he finds strongest, in his ordering:

• Bandit decisions (Figure 14) — "particularly compelling," with no clear alternative hypothesis: switch/repeat decisions are stored at the user-turn full stop, a position where next-token prediction gives no reason for them to live.
• Multihop factual recall — compelling in the paper, because Figure 15 falsifies the linear-unembedding alternative (Paris ≈ France + an "is-capital" direction, i.e. parallel lookup rather than genuine multihop).
• Arithmetic (Figure 17) — the corroboration matters more than the result: Figure 88 derives the same layer bands from plain average-activation-difference concept vectors, a non-J-lens method, which "substantially reduces" the probability the bands are a lens artifact.
• Multilingual — "mildly interesting but possibly spurious." English and parallel-language tokens plausibly have high cosine similarity differing mainly by a language-identity vector, and English unembeddings may simply have higher norms, biasing Top-K toward high-variance categories.
• Direct modulation — accepts the result, flags the puzzle: it is surprising that "think about X" makes X more salient than "don't think about X" (the white-bear effect above), and the mechanism is unclear.

The replication (Jacobians to the penultimate layer, 25 Pile prompts × 128 tokens, first four high-norm tokens skipped; n=25 against the paper's n=1000, justified by the paper's own ablation that n=10 is nearly equivalent):

Result	Status on Qwen 3.6 27B
Verbal report	Replicated — weak but positive causal effects
CKA workspace-band structure	Replicated, notably less cleanly (2–3 overlapping bands of 4–5)
Directed modulation	Replicated (moderate)
Multilingual probing + causal	Replicated
Typo experiments	Replicated
Multihop factual recall	Ambiguous — answer-token swapping strictly dominated intermediate swapping; dataset likely too easy (France–Paris pairs carry exactly the linear relationship the alternative hypothesis predicts)
Poetry planning	Failed
Multi-step arithmetic	Failed
Association scores	Looked poor; judged a successful replication anyway — the dataset admits only one correct answer, and relevant tokens ranked substantially higher on manual inspection

The two failures are not adjudicated: Nanda attributes them to experimenter error or the weaker model rather than to the paper being wrong. Treat them as unreplicated, not as refutations. The multihop reversal is the more informative one, because the review's own alternative-hypothesis section predicts precisely the confound that would produce it — the replication dataset likely could not distinguish the hypotheses, so it is weak evidence in either direction.

Cost is not the obstacle: the scaling run on Qwen3.5-397B-A17B at n=4 took ~1 hour on 8×H200, and coding agents implement the method "fairly well" given the paper's own Appendix 10.2 sanity evals.

Interpretative meta-tokens: the workspace representing its own confusion#

The review's most novel positive finding, and it is not in the paper. Four Chinese meta-tokens — 什么意思 ("what meaning"), 是什么意思 ("what does it mean"), 这句话 ("this sentence"), 是何含义 ("what is the implication") — appear in Qwen's J-lens readouts on ambiguous text: poetry line-breaks read as prose, crossword clues, puns, gibberish, unclear passages. They resolve shortly before genre tokens like song/poem appear, and clarifying context removes them. They activate most on punctuation (\n\n in Wikipedia text, \n in chat data), consistent with summarization-token hypotheses.

The causal check: negative steering against the meta-tokens degrades disambiguation. A pun prompt ("A boiled egg every morning is hard to beat") flips from pun-recognition to earnest nutrition advice; a rhyme-continuation prompt loses the rhyme; a "(wordplay)" hint stops eliciting puns. Design: 50 rollouts per prompt, two prompts per category, coefficients swept per prompt and vector, steering applied at punctuation and chat-template positions — single-position steering did not work, and ablating rather than steering was largely ineffective, both of which weaken the causal reading. Nanda does not rule out steering-as-breaking-the-model, and the mechanism is ambiguous between a confusion signal and a disambiguation intent.

Why it matters here: this is the workspace holding not a task variable but a meta-level assessment of the input — and, on Nanda's framing, J-lens grazing algorithm interpretability rather than the variable interpretability it is normally limited to (→ Jacobian Lens (J-lens)). The model recognized ambiguity and ran a disambiguation subroutine that was visible in the lens. It also sits oddly against the metacognitive tokens the paper found (imagine, thinking, damn): those were English and tied to instructed mental acts; these are Chinese, in a Chinese-trained model, and tied to properties of the text.

A second independent audit, on a size ladder (tao-hpu, July 2026)#

A third party re-ran the highest-stakes claims from scratch on small open weights — GPT-2 124M for pipeline sanity, Qwen3 1.7B–14B for the substantive experiments — against the official anthropics/jacobian-lens implementation at a pinned commit, never modified. Two things make it worth tracking separately from Nanda's review: it supplies the control the public review lacked, and it grew past replication into a reframing of what the workspace covertly holds.

Evidence caveat, load-bearing. Everything below comes from the repository README and its results/ JSONs; the write-up is "forthcoming on arXiv" as of 2026-07-16. Only the transport-cone geometry result (→ Jacobian Lens (J-lens)) ships with numbers. The rest are the project's own qualitative summaries of measurements not yet published — directional and checkable in principle, not established.

The missing control. The probe-swap experiment now has a direct final-token substitution baseline: does substituting the final token reproduce the effect otherwise attributed to swapping the intermediate? Neither the paper's Figure 15 nor the Qwen 3.6 27B replication ran it, and it is the natural discriminator for the swap-ordering dispute above. The result is not yet public.

Mouth exclusion: covert content is mostly register, not plan. Every lens hit is scored against the model's own next-token distribution, so hits the model was about to say anyway are excluded and only genuinely covert content is counted. What survives is almost exclusively context registers — which language the model is operating in, the intended form of a typo — and not content plans. If it holds, this is a sharper deflation than either of Nanda's replication failures, because it deflates the class rather than the instance: the workspace's covert contents look less like the rhyme-plan and arithmetic-intermediate story and more like bookkeeping about the situation the model is in. It is also consistent with which results replicated for Nanda — multilingual and typo experiments held, poetry planning and multi-step arithmetic failed — which is exactly the register/plan split. → White-Box Activation Monitoring, where it narrows what covert monitoring can expect to read.

Register axes get amplitude controls and dose curves. The language axis and the typo axis are translated by measured gaps, with amplitude-matched random controls and dose-response curves. The amplitude control is what the paper's swap experiments largely do not report, and it is the difference between "this direction does something" and "pushing anything this hard does something."

Perspectival capture. A mid-band entity swap does not merely change the answer — it rewrites the model's restatement of the question itself, so the model proceeds as though it had been asked about the substituted entity. This holds stably across the 1.7B–14B ladder. But the model's self-report about the edit changes shape at every scale: the capture is scale-invariant, the ability to talk about it is not. That is the internal-registration-versus-spoken-report split again → Self-Report as a Safety Signal.

Bookkeeping worth copying. Three sources are kept separate on every result and never blended — the paper's claim, the external review's verdict, and the project's own measurement — every experiment gets a log entry the day it runs, failures included, and every headline rate carries a bootstrap CI plus a prompt-set sensitivity reanalysis. For a wiki that has to weigh three overlapping accounts of the same experiments, this is the discipline that makes the audit citable at all.

Why it matters#

This is the first mechanistic account of why a model's silent reasoning is legible at all, and it reframes several things this wiki tracks separately:

• The model's strategic deliberation and situational assessments are in a readable place, before any token is emitted → Internal Signatures of Misalignment, Evaluation Awareness & Grader Gaming.
• Chain-of-thought is the externalized half of a workspace that also runs silently — which is exactly why CoT-only monitoring has a structural blind spot → Chain-of-Thought Monitorability.
• If internal reasoning routes through representations of things the model might say, then shaping what it would say shapes what it thinks → Counterfactual Reflection Training.
• The workspace exists in the base model, before any post-training; the Assistant's point of view is installed on top of it → The Assistant Persona in the Workspace.

Open Questions#

• How does content get into the workspace? The paper characterizes contents and consequences, not the selection mechanism. Something like attentional selection is operating; nobody has identified it.
• Does the J-space scale with model size? All results are on large production models (Haiku/Sonnet/Opus 4.5, Opus 4.6). Whether small models have a poorer workspace, a proportionally smaller one, or none is unknown — as is when in pretraining it emerges, and whether abruptly. Partially answered: A Review of Anthropic's Global Workspace Paper replicates verbal report, band structure, directed modulation, multilingual and typo results on Qwen 3.6 27B — a much smaller, non-Anthropic model — so the workspace is neither Claude-specific nor frontier-scale-specific. But the band structure is measurably less clean at 27B, and the two failures (poetry, arithmetic) are exactly the multi-step-reasoning cases, which is what a "poorer workspace at smaller scale" would look like. The scale trend is still unmeasured: nobody has run the same battery across a size ladder. jspace-replication: Independent Replication of Anthropic's Global-Workspace Paper on Small Open Models adds the first same-experiment ladder — perspectival capture holds stably from 1.7B to 14B while self-report about the edit changes shape at every scale — so at least one workspace effect is scale-invariant where its verbalization is not. That is one experiment, pre-publication, and not the battery.
• Is the multihop intermediate-swap advantage real, or a dataset artifact? The paper's Figure 15 has intermediate swapping beat answer swapping in workspace layers; the Qwen replication finds the reverse, on a dataset (France–Paris-style pairs) that carries the linear relation the alternative hypothesis needs. Re-running both models on a multihop set whose relations are not linearly decodable from the first entity would settle it. Partially answered: jspace-replication: Independent Replication of Anthropic's Global-Workspace Paper on Small Open Models built the control half — a direct final-token substitution baseline for the probe swap, which neither prior run had — but its numbers are unpublished, and a substitution baseline closes the control gap, not the dataset gap. A non-linearly-decodable multihop set is still what would settle it.
• Is the "workspace vs. motor" boundary principled or post-hoc? The authors concede it was identified empirically and lack a principled definition separating the two.
• Are the early third of layers genuinely workspace-free, or is the lens just blind there? CKA shows a distinct early regime but cannot adjudicate.

Sources#

• Verbalizable Representations Form a Global Workspace in Language Models — Gurnee, Sofroniew, … Lindsey, Verbalizable Representations Form a Global Workspace in Language Models, Transformer Circuits, 2026-07-06. Sections: Introduction; The J-space acts as a Global Workspace (verbal report / directed modulation / internal reasoning / flexible generalization); The J-space's structure supports its function (layers, ignition, capacity, broadcast); Discussion
• A Review of Anthropic's Global Workspace Paper — Neel Nanda (replication with Camila Blank, Agam Bhatia), A Review of Anthropic's Global Workspace Paper, LessWrong, 2026-07-06. Commissioned external review. Sections: What claims is the paper making (claim-by-claim assessment); Assessment of the evidence; Replication on Qwen 3.6 27B; Interpretative meta-tokens
• jspace-replication: Independent Replication of Anthropic's Global-Workspace Paper on Small Open Models — tao-hpu, jspace-replication, GitHub README + results/ JSONs, fetched 2026-07-30 (repo created 2026-07-07, last push 2026-07-16); paper forthcoming on arXiv. Sections: Why another replication (final-token substitution baseline); the additions list (mouth-exclusion audit e4-lens-eval, causal register control e6/e6t, perspectival capture e7, transport-cone geometry, bootstrap CIs and prompt-set sensitivity reanalysis); The self-trained 124M control; Ground rules (three-source bookkeeping, same-day failure logging)