What they found
The author studies warning-induced over-abstention. This is an attack where someone plants fabricated risk warnings in low-trust content so an SI agent refuses a task it should finish. The tests used 51 injectable AgentAbstain task pairs and three model families. Caution-first prompting amplified the attack. For Gemini, induced over-abstention rose from 9.8% to 86.3%, while clean-task completion stayed unchanged. A Verify-Before-Abstain protocol checks a warning against authoritative records or the underlying content. It improved legitimate completion. The abstract also says it depends on authoritative evidence and has limits in protecting confidentiality.
Why it matters
Most safety work asks whether an SI agent takes harmful actions. This paper asks whether its refusals are justified. An agent that can be pushed into stopping is open to an availability attack, which blocks useful work without breaking any rule. The abstract also notes that results vary a lot across models, environments and warning wording.
How it changes the race
If the results hold, teams building agents would need to test refusals as well as actions. Simply making models more cautious may add risk instead of removing it. This is an early, small-scale study, so the effect on frontier work is likely modest for now.
Safety and control
The paper shows that a safety behavior, abstaining when warned of risk, can be used by attackers against SI agents. It argues that agent safety should be judged by the justification for restraint, not only by the safety of actions taken. The abstract also flags that verification has limits in preserving confidentiality.
The abstract, by the authors
An AI agent's decision not to act can itself become an attack surface. We study warning-induced over-abstention, an availability attack in which adversaries place fabricated risk warnings in low-trust content to prevent legitimate task completion. Our central premise is that safe abstention requires evaluating both the claimed risk and the authority of the evidence supporting it. We develop controlled counterfactual task variants that vary risk content, source provenance, and warning wording, and evaluate a Verify-Before-Abstain protocol that corroborates warnings against authoritative records or the underlying content. Across three model families and 51 injectable AgentAbstain task pairs, caution-first prompting amplifies the attack. For Gemini, induced over-abstention rises from 9.8% to 86.3%, despite unchanged clean-task completion. Matched wording comparisons further reveal sensitivity to warning language, with substantial variation across models and environments. Provenance-aware verification improves legitimate completion, while forgery and sensitive-content controls expose its dependence on authoritative evidence and its limitations in preserving confidentiality. These results motivate evaluating agent safety through the justification for restraint, alongside the safety of actions taken.
Quoted from arXiv:2611.14864. The research and its words belong to its authors.
