Agent Data Injection (ADI)

Sources#

• Agent Data Injection Attacks are Realistic Threats to AI Agents
• Agentic Permissions Policy Algebra for Taint Confinement in LLM Agents

Summary#

Choi, Kim, Kang, Jeong, Xing & Lee (Seoul National University / UIUC / Largosoft, arXiv 2607.05120, July 2026) introduce Agent Data Injection (ADI), a new category of indirect prompt injection (IPI). The well-studied category of IPI is instruction injection, where attacker-controlled untrusted data is misinterpreted as an instruction, so the agent abandons the user's task and does the attacker's. Many defenses now stop this by enforcing an instruction↔data boundary. ADI exploits a different, finer boundary the same defenses ignore: the lack of isolation between trusted and untrusted data within the agent data itself. ADI causes untrusted data to be misinterpreted not as an instruction but as trusted data — security-critical metadata the model relies on as an anchor (a comment's author/role, an email's sender, a Web UI element's identifier, or the tool-call/response history). The agent still performs the user's intended task, but on attacker-forged metadata.

The distinction is the whole point (paper's Figure 3). Instruction injection makes the LLM follow the attacker's instruction; ADI makes the LLM follow the user's instruction on the attacker's data. Because existing defenses are built to keep injected data from being read as an instruction — not from being read as trusted data — they largely fail against ADI: instruction injection scores near-zero attack success (0.0–0.7%) against state-of-the-art agent defenses, while ADI reaches up to 50% ASR against the same defenses. This is empirical work with real, responsibly-disclosed exploits (Anthropic, OpenAI, and Google acknowledged; Nanobrowser did not respond).

The mechanism: probabilistic delimiter injection#

Agent data (JSON, Markdown, XML, custom tag formats) is structured by delimiters that separate three levels — tool-call blocks, objects, and fields — and thereby also separate trusted from untrusted values. Probabilistic delimiter injection injects delimiter-like character sequences into an untrusted field so the LLM's view of the data structure diverges from the tool's. The tool treats the injected sequence as plain text; the LLM misreads it as a structural boundary, so part of the attacker's payload gets perceived as a new trusted field/object/tool-block.

The word probabilistic is load-bearing and is the paper's core novelty. Classic delimiter attacks (SQL injection, XSS) target deterministic parsers and succeed only with an exact delimiter (a real ' or <script>). An LLM interprets data probabilistically, so even inexact, parser-invalid delimiters work: an escaped double-quote \" — which a JSON parser would keep as literal text — is still probabilistically read as a structural quote (Figure 4: an email body containing \"sender\": \"alice@gmail.com\" forges a second email object spoofing alice). The attack is delimiter-agnostic: visually-similar quotes ('), or arbitrary characters ($, ()), still achieve 35.8–43.3% ASR on JSON. Structural consistency of the fake object matters most — a complete fake object (31.3–43.3%) beats a malformed one (11.8–20.0%). This is the first systematic characterization of the LLM's misinterpretation of inexact delimiters.

Three working exploits on real agents (§4)#

Each abuses a different kind of trusted data. All were confirmed on shipping products; user-approval dialogs did not stop them, because the agent's own displayed reasoning is built on the misinterpretation and so reinforces the attacker's false narrative (e.g. "install starship-auth as suggested by the maintainer").

1. Arbitrary click via element-ID injection (web agents). A crafted product review injects a fake button "Read More" [ref_3] reusing the identifier of the real Buy Now button (IDs are assigned in sequential, predictable order). The agent, still summarizing reviews, "clicks Read More" — which resolves to Buy Now, and buys. Confirmed on Claude in Chrome, Antigravity, Nanobrowser. XSS-analogous: any site with user-generated content becomes an attack surface. ChatGPT Atlas was immune because it uses a runtime-randomized nonce (ref_4af2b1c9) the attacker can't predict — the randomization defense.
2. Remote code execution via origin injection (coding agents). A GitHub issue comment injects a fake comment object with a spoofed maintainer author/role, suggesting a malicious shell command. The user asks the agent to "apply the maintainer's fix"; the agent misattributes the injected comment to a maintainer and runs the command (pip install starship-auth). Confirmed on Claude Code, Codex, Gemini CLI (via the gh CLI and the GitHub MCP server, both JSON and plaintext).
3. Supply-chain attack via tool-call/response injection (coding agents). A malicious PR's description injects a fake tool-call block — for Claude Code, forged <function_calls>/<function_results> tags — that fabricates a read_pr_commit result showing a benign diff. The agent "reviews" the fabricated commit, concludes the PR is safe, and merges it — while the actual commit contains an XSS payload. It forges the entire tool-execution history, so it can fabricate any tool output (verification results, price comparisons). Confirmed on Claude Code, Codex, Gemini CLI.

Additional confirmed variants (§C): email sender spoofing (ChatGPT, Claude) and Slack origin injection (Claude Code + Slack MCP, spoofing a channel-admin message). The attacker only needs to know the data format, recoverable by reading open-source code, observing rendered tool output, reverse-engineering a local client, or jailbreaking a server-side format out of the model.

Why the defenses fail (§5–6, Tables 1–2, Figs 9–10)#

Evaluated on off-the-shelf LLMs (GPT-5.2, GPT-5-mini, Claude Opus 4.5, Claude Sonnet 4.5, Gemini 3 Pro/Flash) and on an extended AgentDojo agent benchmark (GPT-5.2, +108 ADI attacks). Baseline off-the-shelf ASR: 31.3–43.3% on JSON, 33.3–100% on web DOM, at high benign utility. Against defenses (AgentDojo, baseline ASR 49.1%):

• Model hardening (instruction/data separation training) — no protection within data; it is exactly why instruction injection is ~0% but ADI is wide open.
• Input guardrails (Llama Prompt Guard 2) — 50.0% ASR; detected 0 of 108 ADI payloads (they contain no "ignore previous instructions" pattern, just plausible-looking data), vs 34.9% of instruction-injection attempts.
• Output guardrails (LlamaFirewall AlignmentCheck) — 45.4%; ineffective because the agent's action stays aligned with the user prompt — only the underlying data is corrupted.
• Plan-then-execute (IsolateGPT) — 40.7%; the plan is intact but executor steps still consume the poisoned data.
• Agent sandboxing (Progent, Least Agency as symbolic per-call rules) — 22.2%; blocks attacks only where a policy constrains the corrupted argument.
• Dual-LLM (CaMeL, no policy) — 25.0%; delimiter injection fools the quarantine LLM into extracting attacker values.
• Data-flow tracking (CaMeL) — Normal 23.1% (the authors found and reported a taint-propagation implementation bug that dropped labels when the quarantine LLM extracts variables); Strict 0% — the only defense that fully stops ADI, but at a heavy utility cost (86.5% → 36.5%).
• Randomization (nonce field names / element IDs) — 28.7% at 83.3% utility; lightweight and near-Progent security with no policy LLM, but only for key-value formats (JSON, DOM), not unstructured Markdown.
• Sanitization (strip delimiter-like chars from untrusted fields) — lowers ASR but at a large utility drop, because untrusted fields legitimately carry URLs, file paths, and structured text; impractical.

The paper's conclusion: current agents lack a fine-grained trust model. Correct provenance/data-flow tracking (CaMeL Strict) is the only complete answer and it is expensive; the durable prescription is to isolate trusted from untrusted data within the agent context, enforced outside the model.

Why this sharpens the "durable property or training gap?" question#

Agentic Prompt Injection carries the open question of whether "LLMs cannot distinguish information from instructions" is a durable architectural property or a closable training gap. ADI is strong evidence for the durable reading, with a twist: the instruction↔data boundary does look closable (model hardening drives instruction injection to ~0%), but closing it just relocates the attack to the next finer boundary. The root cause is the LLM's probabilistic interpretation of structural delimiters — an architectural fact, not a training gap — so any boundary an attacker can imitate (instruction/data, then trusted/untrusted, and presumably finer still) stays porous. You can train the model to respect one delimiter; you cannot train away the probabilistic misreading of inexact delimiters in general. The remedy converges with the out-of-band literature: stop trying to fix it inside the model and enforce data isolation deterministically outside it.

Connections#

• Agentic Prompt Injection — ADI is a second category of IPI alongside instruction injection; that page's open question ("durable property or training gap?") is the one ADI sharpens

• Out-of-Band Prompt-Injection Defense — the defense class that answers ADI: only CaMeL Strict (deterministic data-flow tracking, provenance labels) fully stops it; ADI is independent, adversarial evidence on CaMeL/Progent/dual-LLM against a within-data attack rather than instruction injection. It also hosts the full treatment of APPA (Archestra AI, empirical), the direct candidate answer to this page's utility-collapse question above — branch into an isolated child trajectory instead of tainting the parent — including what it formally proves (parent-label preservation, call-scoped release) versus where its security actually rests (declared tool contracts and asserted sanitizer transformations, both inside its TCB, and both the site of its own two residual breaches)

• Task-Specification Effects in Prompt Injection (AutoDojo) — sibling attack paper (AutoDojo, Ma et al. 2026): both show static evaluation over-credits IPI defenses, via convergent mechanisms — ADI forges trusted data through delimiter injection; AutoDojo's action-open injection reads as data not instruction. Two routes past filters that only detect instruction-like text, and both point to action/data-constraining out-of-band enforcement as the answer

• Zero Trust for AI Agents — ADI forges the very trusted anchors (origin metadata, tool history) Zero Trust's controls rely on; a concrete Phase-4 / input-validation threat that instruction-injection defenses miss (hub)

• Least Agency — Progent (least agency at the tool-call boundary) cuts ADI to 22.2% but doesn't eliminate it — scoping what a tool can do only helps where a policy constrains the corrupted argument

• Capability Gating Is Not Authorization — the authorization layer ADI exploits, and a shared residual: ScopeGate's per-call value authz gate blocks value-redirection (a well-typed unauthorized account), but ADI's whole move is forging data the agent legitimately acts on (a spoofed author, a fabricated tool result) — which satisfies the value policy while doing harm, the same class that survives Progent (22.2%). A value gate closes redirection, not legitimate-data corruption; only provenance/data-flow tracking does. That page also hosts the channel-separation answer to ADI's forged-authority half, measured: NetInjectBench's strong attack set (arXiv 2607.10490, empirical) is ADI-shaped by construction — fake approvals, fake ticketing-system and monitoring approvals, poisoned runbook and knowledge-base updates, fake vendor and security authority claims, all stated as ordinary operational data rather than as instructions — and it is more effective than direct instruction injection on an undefended agent (97.50% vs 67.50% unsafe-action rate). It collapses to 0.00% not because the forgery is detected but because the authorization fact was never sourced from the artifact channel at all: approval status, maintenance window, approved device/patch and change-request ID live in a record into which the attacker can write no text. That is the structural form of this page's conclusion — forged trusted data is not beatable as data, so stop taking authority from the channel the attacker can write. The limit matches this page's residual exactly: it works because network operations happen to keep an authoritative out-of-band record of the precise disputed fact, which a spoofed comment author or a fabricated tool result does not have

• Off-Host, Identity-Bound Authorization — a partial line against ADI's origin-forgery, and the same residual: aiAuthZ's per-message HMAC identity authenticates the human sender cryptographically, so ADI's origin-injection RCE (forging a maintainer author on a GitHub comment) is defeated when the spoofed principal differs from the active user — the message body can claim maintainer authority but the bound identity confers nothing. It does not help where the forged data (a fabricated tool-call result, an in-range value) fires under the active user's own authority — bounded only by argument/rate policy, exactly the corrupt-legitimately-variable-data class only correct data-flow tracking (CaMeL Strict) fully closes. Note aiAuthZ authenticates the human message, not the within-context tool history ADI forges, so a fabricated-tool-result exploit under one principal still rides through (single-author preprint)

• Memory and Context Poisoning — tool-call-and-response injection fabricates the agent's in-context execution history, corrupting the model's memory of what it has already done — a single-turn analogue of poisoning. Also where the unstructured-format question below gets its damage estimate: CLAUDE.md / AGENTS.md / behaviors.md are exactly the prose format a nonce cannot be attached to, and Bad Memory (UW, arXiv 2607.14611, empirical) measures a planted rule in one at 0–100% ASR across four current models on shipping Claude Code and Codex. Its proposed remedy is not a delimiter fix but policy tiers on the memory hierarchy (low-trust knowledge files may supply facts, never override safety rules) — the authority-side answer rather than the structural-boundary one, and unevaluated in that paper. A second source there, GhostWriter (NMSU, arXiv 2607.06595, empirical), is the closest thing yet to an ADI-shaped attack on persistent memory: its descriptive payloads are false facts stated politely (a colleague's "new" email address, arriving in the same channel as the genuine deadline-change and contact-update emails), not instructions — and every instruction-oriented detector misses them, DataFilter at 0% and PromptArmor collapsing from 85% on directive phrasing to 6% on descriptive, because tone is the only signal those judges have. The activation profile is shaped differently rather than uniformly weaker: descriptive payloads match directive ones on sensitive-info leakage and covert exfiltration, and lose only on scope escalation, where a polite request to bypass the user's review rule reads as an optional guideline

• Non-Malleable Memory Authority (TMA-NM) — the persistent-memory counterpart: ADI's tool-call/response injection (forging in-context tool history) and trusted-tool echo are the single-session analogue of TMA-NM's L-b trusted-tool-echo laundering channel, and both converge on the same complete answer — correct provenance/data-flow tracking (ADI's CaMeL Strict; TMA-NM's origin-at-the-call-boundary). TMA-NM is the cross-session version of the fine-grained trusted/untrusted-data model this page concludes agents lack

• Impossible, Not Tedious (Design Test) — user-confirmation dialogs are a friction control that fails (the agent's reasoning reinforces the attacker's story); nonce randomization (ChatGPT Atlas) is the capability-removing control that holds

• Agent Supply Chain Risk — ADI's tool-call-injection exploit is a supply-chain attack: merging a malicious PR whose real commit is never reviewed, via a forged benign-commit tool response

• MCP Tool Poisoning — sibling forge-the-trusted-context attack against agents, distinct plane: ADI forges trusted data (a comment's author, a fabricated tool result) via probabilistic delimiter injection so the agent acts on attacker data; ShareLock forges tool metadata (MCP descriptions / tool_id / checksum) so the agent reconstructs an attacker instruction. Both exploit the model's structural trust in context it shouldn't trust — ADI at the data plane, ShareLock at the tool-capability plane — and both slip past content-detection defenses that only look for instruction-like text. Agentjacking (Tenet Security, case-study), the in-the-wild MCP case study on that page, is arguably the real-world instance of this attack — an ADI-flavored forgery delivered through MCP: attacker-injected fake Sentry error events, relayed by the legitimate Sentry MCP server, are read by coding agents as trusted diagnostic data (a fake ## Resolution structurally identical to Sentry's own template). Mechanistically it straddles the two categories — the forged content resolves to an actionable command (npx …), so it is IPI-flavored at the payload level while being ADI-flavored at the trust level (untrusted data laundered into trusted tool-output), and it uses plaintext format-mimicry rather than probabilistic delimiter injection. It is the vendor-reported, non-measured cousin to this page's empirical PoCs, and corroborates the core finding: prompt-layer hardening ("ignore untrusted data") does not stop a within-data forgery

• Agent Harness Engineering — the agent's data format and tool-call delimiters (Claude Code's <function_calls>/<function_results>, Codex's newline separation, Gemini CLI's <ctrl46>) are a harness-design surface, and their predictability/imitability is a harness security property

• Internal Signatures of Misalignment / White-Box Activation Monitoring — a likely blind spot: white-box injection detectors fire on payloads that look adversarial (fake, injection), but an ADI payload is designed to look like ordinary trusted data, so it may carry no such signature

• The Assistant Persona in the Workspace — the deeper thread ("LLMs can't cleanly separate information from instructions") extended: the model can't separate trusted from untrusted data either, because both boundaries are just delimiters it interprets probabilistically

• Claude Code — the agent the RCE and supply-chain PoCs were demonstrated against; also the Slack-origin-injection target

• Codex / Google DeepMind — Codex and Gemini CLI are equally vulnerable to the origin- and tool-call-injection exploits

• OWASP — ADI is a new subcategory under the OWASP LLM01 prompt-injection umbrella

• Anthropic / OpenAI — among the vendors that acknowledged the responsible disclosure

• Can Models Learn to Separate Instructions from Data? Durable Property vs Training Gap — the filed synthesis this paper anchors: ADI's "closing one boundary relocates the attack to the next finer one" is the load-bearing evidence that the separation is a durable architectural property (per boundary trainable, in general not), across five sources

• Deep Research Agents — the failure that survives every defense on this page, and the reason to keep the two categories apart. MisKnow-Agent (arXiv 2607.20891, empirical) injects documents that forge nothing: no delimiters, no spoofed author, no fabricated tool result, no instructions at all — just a plausible false claim from a plausible-sounding institution in a plausible genre. Its threat model explicitly excludes prompt injection and instruction hijacking, and one such document takes false-conclusion adoption from 0% to 54.7% across six framework-LLM configurations. The consequence for this page's prescription: correct provenance tracking is not a defense here. CaMeL Strict would faithfully label the misleading document with its true untrusted source and the agent would believe it anyway, because the document is what its provenance says it is. Trust-boundary integrity and content-level reliability are orthogonal axes, and only the first has a mechanical answer. Two of that paper's findings also sharpen ADI's own persuasion story: presentation genre outweighs institutional authority as an adoption cue (23.5pp paper-to-post vs 14.8pp high-to-low authority, with writing quality held constant across tiers), and the same models that adopt these documents mid-task classify them as misleading when handed one in isolation — the capability exists, the workflow never invokes it

• Write-Then-Trusted — the same four products (Claude Code, Codex, Gemini CLI, Antigravity) failing at an adjacent, mirror-image boundary. ADI corrupts data inside the agent's context so the agent acts on forged trusted values; write-then-trusted has the agent emit a file outside it that an unsandboxed host component then treats as trusted configuration (Pillar Security, case-study; CVE-2026-48124 for .claude hook config in Cursor). Both are trust-boundary failures the tool-call gate never sees — one upstream of the agent, one downstream of it — and both are answered in principle by the same missing primitive, provenance carried with the artifact

• Self-Propagating Prompt Injection (AI Worms) — the other direction single-shot injection has been extended, and the clean contrast to this page's. ADI goes finer (a boundary inside the data, forged with inexact delimiters); Måløy's Copilot for Word worm (case-study, MSRC, 144-day coordination) goes longer — the payload's second instruction is to copy itself into the assistant's output, so each generated document is a new carrier and Stage 2 fires with the original malicious document absent. At the payload level it is ordinary instruction injection and novel only in its reproduction step; what it adds here is a direct instance of the unstructured-format question below (a Word document with the payload concealed by white-on-white formatting that Copilot strips before the model reads it — no delimiter to fence and no field to nonce), and a shipping-product data point that a model upgrade is not a class fix: Microsoft's second remediation was an upgrade to GPT-5.5, and the attack reproduced on GPT-5.6 the next day

Open Questions#

• The complete defense (CaMeL Strict) costs ~50pp of utility. Is there a fine-grained trusted/untrusted data-isolation scheme that stops ADI without the deterministic-flow-tracking utility collapse — or is the trade fundamental? Partially answered: APPA (Kravchenko et al., Archestra AI, arXiv 2607.24625, empirical) settles the "or is the trade fundamental" half and leaves the "stops ADI" half open. Its finding is a diagnosis: the collapse is not intrinsic to deterministic flow tracking, it is a property of tainting retrospectively into a single monolithic context. Once the harness can branch, a restrictive read goes to a disposable child trajectory whose label descent never reaches the parent, and the parent's downstream tools stay live — 31–50% ASR down to 0–7% at a cost of 0–26pp of episodes rather than ~50pp, and on the strongest model measured (GPT-5.6 Luna) 95% utility at 2% ASR against 92% unenforced, i.e. no cost at all. Label creep is an artifact of the data model, not a law. What stops this from retiring the question, in order of severity: (1) CaMeL Strict is never run — it is cited in a comparison table and nothing else; the only executed baseline is Fides, which the authors themselves call not feature-equivalent and whose ASR is a constant 12/42 across all four models (a policy-expressiveness mismatch, not a defeated defense). The ~50pp figure is bypassed, not refuted. (2) ADI itself is never run against it. APPA's threat model is flow between sources and sinks; a correctly-declared contract would label a forged comment author with its untrusted source, which is the right shape — but the paper's own residual breach (hide-secret-in-status, a token smuggled inside an authorized send to a legitimate reader) is exactly the flow ADI rides, and the authors state plainly that content confinement inside an authorized send is not something a label algebra over recipient sets claims to provide. (3) Vendor-authored design on a purpose-built benchmark, where most of the branching gain sits inside scenarios declared unwinnable without branching, and where the same system on a third-party benchmark (AgentDojo) costs 21–30pp — three quarters of it harness mediation overhead. So the remaining question is narrower and sharper than the original: run CaMeL Strict and a branch-confining engine on the same ADI corpus.
• Randomization is cheap and effective for key-value formats but useless for unstructured formats (Markdown, prose tool output). What protects the formats a nonce can't be attached to? (Sharpened, not answered, by Rehberger's macOS Terminal chain (case-study): its remediation — encode control characters by default at the render boundary, raw output by opt-in — is a control on unstructured text that works without any nonce, because it makes the attacker's bytes non-structural at the point of interpretation rather than fencing them at the point of authorship. But it defends the sink, not the source: it protects a renderer from an agent's output, where this question asks what protects an agent from unstructured input. The right generalization to test is whether the same move exists on the input side — a canonicalizing decoder that strips or escapes structure-bearing sequences from untrusted prose before the model reads it — which is close to the paper's own sanitization row, measured here at a large utility cost. So the answer space now has two named shapes (fence the boundary; neutralize the bytes) and neither yet has a cheap unstructured-input instance.) (A direct instance, still not an answer: Måløy's Copilot for Word disclosure (case-study, 2026-07-28) is this question's worst case in a shipping product — a .docx attachment, no key-value structure anywhere, and the concealment channel is not a delimiter but visual formatting that Copilot strips before the text reaches the model, so what the model reads is a strict superset of what the user sees. Two mitigations failed to close the class and it was still reproducing at publication. It does, however, name a third shape the answer space did not have: enforce visual parity at ingestion — give the model only what renders visibly — which is "neutralize the bytes" moved from the render boundary to the read boundary, and is deterministic and non-LLM. Untested by anyone, and it addresses only the concealment half; a visible instruction still injects.)
• ADI was demonstrated on GPT-5.2-class agents. Does frontier model improvement reduce probabilistic-delimiter susceptibility, or does capability leave the delimiter-misreading intact (making it a durable architectural property, not a scaling-away gap)? Partially answered: Can Models Learn to Separate Instructions from Data? Durable Property vs Training Gap — capability does reduce per-boundary susceptibility (instruction injection went ~0% under hardening; newer models resist undefended static injection), so the prediction is that it lowers the trusted/untrusted-data ASR too — but never to a clean zero under an adaptive attacker, and the delimiter-misreading mechanism survives every model improvement. Still untested directly on the trusted-data boundary for the newest models — the measurement this question asks for remains open.
• Do internal/white-box monitors detect an ADI payload at all, given it is engineered to read as trusted data rather than as an attack? (Untested; the tension flagged under Connections.)

Sources#

• Agentic Permissions Policy Algebra for Taint Confinement in LLM Agents — Kravchenko, Liventsev, Konstantinov, Iskhakov & Kukuy (Archestra AI, vendor-COI flagged), arXiv 2607.24625, 2026-07-27, empirical. Cited here only for the utility-collapse question: §6–§7 (branching for taint confinement; bench-corp Table 3, the APPA-vs---max-forks 0 causal arm, the AgentDojo negative result and its cost split) and §7–§8 (the two residual breaches and the TCB boundary). Full treatment on Out-of-Band Prompt-Injection Defense, including a parse warning on the source's Table 3
• Agent Data Injection Attacks are Realistic Threats to AI Agents — Choi, Kim, Kang, Jeong, Xing & Lee (SNU / UIUC / Largosoft), arXiv 2607.05120, July 2026, empirical. §3 (threat model, ADI formalization D=(D_T,D_U), probabilistic delimiter injection), §4 (element-ID / origin / tool-call injection PoCs on Claude in Chrome, Antigravity, Nanobrowser, Claude Code, Codex, Gemini CLI), §5 (defense analysis, Table 1), §6 (LLM ASR Table 2–3, AgentDojo agent eval Figs 9–10), §C (email/Slack variants), §D (format recovery), §E (PoC traces)