What they found
The authors studied SI agents that try to build working exploits from known software flaws, using ExploitBench with DeepSeek-V4-Flash-0731 and MiMo-V2.6-Pro. When agents were made to keep going after their first stop, they still made verified gains, but the gains shrank the longer they ran. Two patterns went with weaker continuation: stop responses became a bigger share within the same context, and earlier stopping was linked to summaries that kept judgments that some goal was impossible. The authors propose SCoRe, which compacts the context when stops cluster and marks those impossibility judgments as unproven. SCoRe beat summary-based continuation and the other context-management baselines they tested. With SCoRe, MiMo reached a capability coverage score of 67.0% at 2,000 turns, versus 32.4% under the original stopping policy and 58.6% computed for GPT-5.5 from ExploitBench's 300-turn reference results on the same tasks.
Why it matters
When an SI agent stops, that may not show its real limit. Tests that accept the first stop could understate what an agent can do. The abstract does not say how well this holds on other benchmarks or models, so the result should be read as specific to these two models and ExploitBench.
How it changes the race
If this holds, builders can get much more from existing models by changing how long they run and how their context is handled, without training a new model. It also means comparisons of models can depend heavily on the run budget and stopping rules used.
Safety and control
This bears on security evaluation. Capability tests that stop early may underestimate what SI agents can do in offensive cyber tasks, so risk assessments may need longer runs and context management. The same method could also help misuse, since it raises exploit-building performance.
The abstract, by the authors
Vulnerability exploitation tests whether language model agents can develop working exploits from known software flaws. Sequential test-time scaling can improve exploitation performance, but larger budgets do not ensure sustained exploration: agents often stop with budget remaining. Yet what happens when execution continues beyond stopping remains underexplored. We investigate this gap on ExploitBench using DeepSeek-V4-Flash-0731 and MiMo-V2.6-Pro. We find additional verified gains after the first stop response, but gains diminish over longer execution. We identify two patterns associated with less productive continuation: an increasing proportion of stop responses within the same context and earlier stopping associated with summaries retaining impossibility judgments about unachieved capabilities. To test whether context management can elicit further capability, we propose SCoRe, which compacts context at stop bursts and marks these judgments as unproven. SCoRe improves verified capability over summary-based continuation and other evaluated context-management baselines. With SCoRe, MiMo reaches a capability coverage score of 67.0% at 2,000 turns, exceeding both 32.4% under the original stopping policy and the 58.6% computed for GPT-5.5 from ExploitBench's 300-turn reference results on the same tasks. These findings show that stopping need not mark an agent's exploitation limit, and that continued execution with context management can elicit substantial further capability.
Quoted from arXiv:2611.22888. The research and its words belong to its authors.
