Task-Specification Effects in Prompt Injection (AutoDojo)

AutoDojo (Ma et al., arXiv 2606.15057): a cheap black-box adaptive attack that iteratively optimizes an indirect prompt injection against a live defended agent using only the success/fail signal — recovering 28% overall ASR (64% on action-open tasks) against a filter that scores 0% *static* ASR, so static-benchmark robustness dramatically overstates real robustness; plus the task-specification axis it exposes — under-specified 'action-open' tasks (the user defers the action itself to attacker-reachable content) are markedly more injectable than fully-specified ones for prompt- and filter-based defenses, while action-constraining system-level defenses invert this and grow stronger