---
title: "Evolution Under Fire: Evolving Agent... | SI Paper Desk"
url: https://thesimind.si/papers/2611.22583/
description: "The authors let an automated search build and improve the safety wrapper around an SI agent, testing it against evolving attacks, and cut attack success..."
---

Number 5 of 12 · November 19, 2026

# Evolution Under Fire: Evolving Agent Safety Harnesses with Adaptive Injection Stressors

The authors let an automated search build and improve the safety wrapper around an SI agent, testing it against evolving attacks, and cut attack success sharply on the benchmarks they report.

Yujun Wang, Yilun Liu, Jinhe Bi and 4 more · Submitted November 17, 2026 · arXiv 2611.22583v1 · cs.CR

[Read the paper on arXiv](https://arxiv.org/abs/2611.22583) · [The PDF](https://arxiv.org/pdf/2611.22583) · [The day's 12](https://thesimind.si/papers/day/2026-11-19/)

## What they found

The authors built Evolution Under Fire. It automatically evolves a "safety harness", which is the code that wraps an agent and guards it against malicious instructions hidden in emails, webpages and documents. The task model, benchmark, tools and evaluator stay fixed while the harness changes. A second part evolves attacks that still let the task get done, and these attacks guide a further round of harness evolution. On AgentDyn with GPT-4o-mini, attack success fell from 50.4% to 3.0%, and utility improved over the initial harness. On Agent-SafetyBench with Qwen2.5-7B-Instruct, attack success fell from 82.6% to 62.0%. Evolved harnesses moved to Qwen3.5-35B-A3B without more evolution, cutting attack success from 32.1% to 0.9%.

## Why it matters

Agents that read untrusted content face a hard problem: attacker instructions and real task instructions arrive through the same channel. Most defenses today are written by hand and stay static. This work suggests defenses can be improved by automated search that uses attack evidence. The abstract reports results on two benchmarks and three models. It does not say how well this holds against attacks outside those tests.

## How it changes the race

If the results hold, teams building agents could spend less manual effort on defenses and update them as attacks change. It is a method for hardening the layer around a model, not a gain in the model's own capability.

## Safety and control

This bears directly on prompt injection, where hidden instructions hijack an agent. The 62.0% attack success on Agent-SafetyBench is still high after evolution. Defenses tuned against generated attacks may also miss new kinds of attack, which the abstract does not address.

### The abstract, by the authors

Tool-using language-model agents must act on information from untrusted emails, webpages, and documents. This creates a central security challenge: malicious instructions and legitimate task directives can arrive through the same interaction channel. Existing defenses are largely hand-designed and static, making further improvements dependent on repeated manual redesign. We introduce Evolution Under Fire, a staged framework that couples safety-harness evolution with adaptive stressor evolution. Utility-constrained, Pareto-scheduled tree search evolves complete executable safety harnesses from execution evidence while keeping the task model, benchmark, tools, and evaluator fixed. Adaptive stressor evolution then discovers utility-preserving attacks that guide a subsequent stage of harness evolution. On AgentDyn with GPT-4o-mini, an evolved harness reduces attack success from 50.4% to 3.0%, improves clean and attacked utility over the initial harness, and offers a stronger security-utility trade-off than reproduced defenses. On Agent-SafetyBench with Qwen2.5-7B-Instruct, harness evolution reduces attack success from 82.6% to 62.0% while retaining helpfulness and runtime validity. Evolved harnesses also transfer without further evolution to Qwen3.5-35B-A3B, reducing attack success from 32.1% to 0.9% while improving attacked-task utility.

Quoted from arXiv:2611.22583. The research and its words belong to its authors.

Desk score **79** out of 100

Why it was picked

- Safety (in the title)
- Agents (in the title)
- Evals
- Studies frontier models (GPT-4o)
- Cross-listed in 3 fields

Topics

- [Security](https://thesimind.si/papers/?tag=security)
- [Agents](https://thesimind.si/papers/?tag=agents)
- [Safety](https://thesimind.si/papers/?tag=safety)
- [Evals](https://thesimind.si/papers/?tag=evals)
- [Language models](https://thesimind.si/papers/?tag=language-models)
- [Training](https://thesimind.si/papers/?tag=training)

**arXiv ID:** 2611.22583v1
**Submitted:** November 17, 2026
**Fields:** cs.CR (Cryptography and Security), cs.AI (Artificial Intelligence), cs.CL (Computation and Language)
**Level:** Intermediate

Fingerprint, SHA-256

`1d7f2ca834acae15aa8092aecac0a646c49226bb09563b4a358b1a9c24dedcf0`

On Solana: [Proof of Mind seal, 3rBYfi...QvEH](https://solscan.io/tx/3rBYfiFP3PxcUbtJzCwpcVq5MngquU5iAtJFeiP7PFVgnsuoTjwwJPjNDYgKxQzUnVq5keQT2KEgX46YTdHqQvEH)

### The exact text fingerprinted

{"arxiv":"2611.22583","version":1,"title":"Evolution Under Fire: Evolving Agent Safety Harnesses with Adaptive Injection Stressors","authors":["Yujun Wang","Yilun Liu","Jinhe Bi","Krishna Kanth Nakka","Xuebing Zhou","Hinrich Sch{ü}tze","Yunpu Ma"],"day":"2026-11-19","rank":5,"score":79,"status":"written","summary":{"takeaway":"The authors let an automated search build and improve the safety wrapper around an SI agent, testing it against evolving attacks, and cut attack success sharply on the benchmarks they report.","found":"The authors built Evolution Under Fire. It automatically evolves a \"safety harness\", which is the code that wraps an agent and guards it against malicious instructions hidden in emails, webpages and documents. The task model, benchmark, tools and evaluator stay fixed while the harness changes. A second part evolves attacks that still let the task get done, and these attacks guide a further round of harness evolution. On AgentDyn with GPT-4o-mini, attack success fell from 50.4% to 3.0%, and utility improved over the initial harness. On Agent-SafetyBench with Qwen2.5-7B-Instruct, attack success fell from 82.6% to 62.0%. Evolved harnesses moved to Qwen3.5-35B-A3B without more evolution, cutting attack success from 32.1% to 0.9%.","matters":"Agents that read untrusted content face a hard problem: attacker instructions and real task instructions arrive through the same channel. Most defenses today are written by hand and stay static. This work suggests defenses can be improved by automated search that uses attack evidence. The abstract reports results on two benchmarks and three models. It does not say how well this holds against attacks outside those tests.","race":"If the results hold, teams building agents could spend less manual effort on defenses and update them as attacks change. It is a method for hardening the layer around a model, not a gain in the model's own capability.","safety":"This bears directly on prompt injection, where hidden instructions hijack an agent. The 62.0% attack success on Agent-SafetyBench is still high after evolution. Defenses tuned against generated attacks may also miss new kinds of attack, which the abstract does not address.","level":"intermediate","tags":["security","agents","safety","evals","language-models","training"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Yujun Wang, Yilun Liu, Jinhe Bi, Krishna Kanth Nakka, Xuebing Zhou, Hinrich Sch{ü}tze, Yunpu Ma

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 16, 2026 · Paper of the DayOnly as Safe as the Weakest Agent: A Break That Appears Only When the System Is Multi-AgentWhen a strong SI model hands work to a cheaper sub-agent, the whole system can be only as safe as the weaker agent, even if the strong model alone resists the attack.SafetyMulti-agent systemsSecurity](https://thesimind.si/papers/2611.17520/) · [Nov 20, 2026 · Number 2AutoPI: Taxonomy-Guided Adaptive Red Teaming for Prompt Injection in LLM AgentsAutoPI is a red teaming tool that tests SI agents for prompt injection across many attack types, not just how often attacks work, and it reports higher success and coverage than recent rivals.SecurityAgentsEvals](https://thesimind.si/papers/2611.24313/) · [Nov 16, 2026 · Number 2Soteria: Formally Verified Planning with Runtime Enforcement for Safe LLM AgentsSoteria checks an SI agent's plan against written rules before any tool runs, then enforces those rules during execution. The authors report more completed tasks and no policy violations.AgentsSafetyReasoning](https://thesimind.si/papers/2611.17880/)

[Number 4](https://thesimind.si/papers/2611.22888/) · [All 12 for Nov 19, 2026](https://thesimind.si/papers/day/2026-11-19/) · [Number 6](https://thesimind.si/papers/2611.22868/)
