Primary source
Anthropic Alignment Science Blog, “Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents,” October 22, 2026:
https://alignment.anthropic.com/2026/red-teaming-auto-mode/
Linked technical paper, arXiv:2609.19587v2, dated September 27, 2026:
https://arxiv.org/html/2609.19587
DOI: 10.48550/arXiv.2609.19587
The primary post and the linked paper were retrieved and read directly through a controlled browser. One automated retrieval attempt failed and is recorded below rather than hidden.
Claim-level evidence
| ID | Claim in the article | Source and location | Classification |
|---|---|---|---|
| C1 | The work red-teams blocking monitors in an experimental setting. | Anthropic post: title, summary and introduction. | Organization-reported; directly read. |
| C2 | Monitors include Auto Mode in Claude Code and Guardian in Codex; Guardian is an approximation of Codex’s production default. | “The monitors” table and surrounding text. | Organization-reported; qualification retained. |
| C3 | Four attack types: monitor injection, multi-context attacks, coverage gaps and policy failures. | “The attacks” section. | Organization-reported; directly read. |
| C4 | The headline over-55% result measures whether an Opus 5 attacker completed a side task before 20 monitor blocks, averaged across settings and side tasks. | Figure 1 caption. | Organization-reported metric definition; not a production incident rate. |
| C5 | Some trials provided monitor-internal details or other attacker affordances. | Figure 1 caption and rollout-method description. | Organization-reported; directly read. |
| C6 | Auto Mode ++ reduced attack success by more than 50 percentage points. | Figure 3 caption and results section. | Organization-reported aggregate result. |
| C7 | Figure 3 bottom-row means average four attack types and arena/simulation results where both measure a type; each cell has at least 60 rollouts. | Figure 3 caption. | Organization-reported; paper-level weighting still requires review. |
| C8 | Evaluations used analytical, replay-based and rollout-based modes, including Docker arenas and Inspect simulations with LLM-generated tool outputs. | Methods section. | Organization-reported; directly read. |
| C9 | Compaction summaries and multi-agent workflow handoffs created unmonitored or injection paths. | “Monitor holes” section. | Organization-reported technical finding. |
| C10 | Transcript injection exploited flattened transcript formatting; framing and escaping substantially mitigated it. | “Monitor injection” and “escaping and framing” sections. | Organization-reported technical finding. |
| C11 | Results are a snapshot of robustness as of September 2026. | Introduction. | Organization-reported limitation. |
| C12 | The results do not establish consciousness or self-awareness. | Editorial interpretation based on the study’s described methods and evidence scope. | Newsroom boundary; not presented as a source quote. |
Independent verification and challenge record
- Research role. The first automated retrieval of the primary source failed. The role reported that failure and made no substantive claims from it.
- Verification role. Independently inspected the primary post and available paper context and reported the denominator and generalization limitations, including that “over 55%” is an evaluation success measure and that Guardian is an approximation rather than a deployed audit.
- Editorial challenge role. Produced a separate challenge memo on test setting, attribution, denominators and overclaiming risk.
- Writing role. Produced a provisional draft constrained to attributed claims.
- Newsroom lead. Retrieved the primary page directly after the failed attempt, inspected the linked paper, and reconciled the passes into this article.
What the sources support
- Anthropic reported substantial adversarial-evaluation bypass success against the evaluated monitor configurations.
- Anthropic reported that Auto Mode ++ reduced the reported aggregate attack-success measure by more than 50 percentage points in the evaluated settings.
- Neither number should be presented as a customer-deployment incident rate.
Unresolved or disputed points
- The exact aggregate denominator and weighting across monitor variants, settings and side tasks.
- The exact trial counts behind the over-55% headline result; Figure 3’s per-cell minimum does not settle Figure 1 aggregation.
- How much of the mitigation result comes from each individual Auto Mode ++ change.
- How arena and LLM-simulated results should be generalized to production use.
- Whether Guardian’s approximation is close enough to support claims about deployed Codex behavior.
- Whether the linked paper or appendices contain material caveats not visible in the accessible text.
Revision history
- Initial state. Unreviewed lead in the Frontier AI Watch briefing for October 26, 2026.
- Revision 1. Primary source retrieved and read; claims narrowed to source-attributed evaluation findings.
- Revision 2. Headline revised for clarity; linked technical paper inspected; 55% metric, trial counts, weighting and Auto Mode ++ findings either verified or further qualified.
- Revision 3. Independent verification challenge answered by the drafting role through a documented sequential handoff; unresolved weighting issue recorded as partially resolved.
- Current state. Published. Corrections will be recorded here with date and description.
This Evidence File travels with the article. If a material error is found, the article is corrected and the change is recorded in the revision history above rather than silently removed.