What they found
The authors built Learning to Coordinate (L2C). It sets up a team of SI agents to test a language model for safety failures, which is called red teaming. Each test case gets its own workflow, stored as a versioned graph of agent roles, dependencies and inputs. A controller trained with supervised fine-tuning reads records of what was passed between agents, plus the target's reply, the evaluator's verdict and the budget left. It then decides whether to keep the workflow, refine instructions, restructure the team, or lock further edits. The tests used HarmBench with three execution backbones and two target models. In all six executor-target setups, L2C improved attack success rate (ASR) over EvoSynth by 1.4 to 4.5 percentage points. It also cut mean online tokens by 35.6% to 37.3%. The authors also report ablations, dataset transfer and evaluator checks, but the abstract gives no details of those results.
Why it matters
Red teaming is how builders look for safety failures before release. If it costs less to run, more teams can test more often. The abstract says the controller links feedback to specific changes in how agents work together. That is a step beyond only seeing whether an attack worked. The ASR gains are modest, and the abstract does not say how they hold up outside the tested setups.
How it changes the race
If the result holds, safety testing of frontier models gets cheaper, so smaller labs and outside auditors could run more of it. The gain is incremental over EvoSynth, mainly in cost.
Safety and control
This work is meant to help find safety failures in language models, so it supports safety evaluation. Cheaper and stronger automated attacks could also help people who want to misuse models. The abstract does not discuss this.
The abstract, by the authors
Automated red teaming is essential for identifying safety failures in large language models (LLMs), but its practical scalability depends on the resources required to discover those failures. Multi-agent systems can combine complementary capabilities, yet repeated planning, redundant exchanges, and ineffective information routing can make their coordination expensive. Moreover, a target model's response indicates the outcome of a test without identifying which collaboration decision should change. We present Learning to Coordinate (L2C), an LLM-guided framework that evolves an executable multi-agent workflow for each red-teaming case. L2C represents agent responsibilities, execution dependencies, and input bindings in a versioned graph, and associates each graph with records of the information actually delivered and produced. A controller trained through supervised fine-tuning integrates these records with target responses, evaluator verdicts, and remaining budget to retain the workflow, refine instructions, restructure collaboration, or lock further editing. This design connects feedback to explicit changes in subsequent execution while allowing a useful organization to continue without repeated controller intervention. The evaluation protocol covers HarmBench with three execution backbones and two target models, together with component ablations, dataset transfer, and evaluator checks. Across the six executor-target configurations, L2C improves ASR over EvoSynth by 1.4 to 4.5 percentage points while reducing mean online tokens by 35.6% to 37.3%. L2C connects adaptive collaboration to the resource requirements of automated security evaluation.
Quoted from arXiv:2611.24221. The research and its words belong to its authors.
