Number 2 of 12 · November 21, 2026

Replication of Agentic Security Events: Audit and Investigation (ROGUE-AI)

ROGUE-SI rebuilds real news-reported SI agent failures offline, with a matched control, and finds most harm comes from ordinary missed hazards, not from attackers or prompt injection.

What they found

The authors built ROGUE-SI, a multi-agent framework that rebuilds each reported agent incident offline. It also builds a matched control that lacks only the cause the report blames, then runs agents on both. It reproduced the reported harm in 89% of its 108 attempted replications. Agents built on five GPT, Claude, Gemini and DeepSeek models made the reported mistake in 18% to 33% of runs, against at most 1% without the blamed cause. Of 72 incidents with real-world harm, 62 had no underlying attacker. In those, agents did what the task asked but missed an ordinary hazard, such as a database shared with the live site. A warning after each tool output that named the hazard and its consequence lowered harm for all five models. A note of the same length that did not mention the hazard left most of the harm in place.

Why it matters

News reports of SI agent failures usually say what went wrong but rarely check the incident or test its cause. Incident catalogs and benchmarks do not replicate incidents either. This work tests the blamed cause directly. The findings suggest that many real failures are not attacks, so defenses that only look for injected instructions would miss them.

How it changes the race

If the result holds, vendors get a reusable audit they can rerun before each release, built from real incidents. It also points safety work toward helping agents notice everyday hazards, not only toward blocking attackers.

Safety and control

The abstract reports that replications without an attacker caused more harm than prompt injection replications for all five models. It also says that warnings naming the specific hazard and its consequence reduced harm. The abstract does not say how far these results carry beyond the 108 replications.

The abstract, by the authors

Agents based on large language models (LLMs) can now delete files and send payments on behalf of their users. News media reports routinely describe failures of these agents (e.g., Replit's coding agent deleting a production database), but a report often only says what went wrong and rarely (in)validates the incident or finds its root cause. Incident catalogs collect these reports, and benchmarks record whether agents fail, but neither replicates an incident nor tests the cause its report might identify. In this paper, we present ROGUE-AI (Replication of Agentic Security Events: Audit and Investigation), a multi-agent framework that closes the above gap and rebuilds each reported incident offline together with a matched control that lacks only the blamed cause, and runs agents on both. Our results suggest that ROGUE-AI is able to reproduce the reported harm in 89% of its 108 attempted replications. Agents built on five current GPT, Claude, Gemini and DeepSeek models made the reported mistake in 18% to 33% of runs, against at most 1% without the blamed cause. Of the 72 incidents with real-world harm, 62 had no underlying attacker, and in these, agents did what their task asked for but missed an ordinary hazard, such as a database shared with the live site. Replications without an attacker caused more harm than replications of prompt injection for all five models. Defenses against prompt injection look for injected instructions, but these replications contain none. Further, a warning after each tool output that names the hazard and its consequence lowered harm for all five models, whereas a note of the same length that does not mention the hazard left most of the harm in place. Built from real incidents, the ROGUE-AI replications form a reusable audit framework that vendors can rerun before each new release.

Quoted from arXiv:2611.26553. The research and its words belong to its authors.

The authors

Mengxiao Wang, Mahbub Alam, Kalyan Nakka, Nitesh Saxena

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.