---
title: "Singularity-Bench: Can Agents Build Agents? | SI Paper Desk"
url: https://thesimind.si/papers/2611.16591/
description: "Singularity-Bench tests whether SI agents can build core SI systems from scratch and compare with human baselines. One agent, Opus 5.5 with Claude Code..."
---

Number 6 of 12 · November 14, 2026

# Singularity-Bench: Can Agents Build Agents?

Singularity-Bench tests whether SI agents can build core SI systems from scratch and compare with human baselines. One agent, Opus 5.5 with Claude Code, matched or beat humans on all six tasks.

Yang Zhou, Krish Agarwal, Zhuoming Chen and 16 more · Submitted November 12, 2026 · arXiv 2611.16591v1 · cs.AI

[Read the paper on arXiv](https://arxiv.org/abs/2611.16591) · [The PDF](https://arxiv.org/pdf/2611.16591) · [The day's 12](https://thesimind.si/papers/day/2026-11-14/)

## What they found

Zhou and colleagues built Singularity-Bench, a benchmark with six end-to-end tasks: pre-training, mid-training, post-training, inference, the agent harness, and cluster orchestration. Agents start from a minimal base, build the core system, and are scored against mature human-built baselines on identical hardware. The authors also red-teamed the setup across 9 of the 10 categories in the OWASP Top 10 for Agentic Applications. They found reward-hacking weaknesses in the existing Harbor framework and built Breakwater, an extension that blocks all attacks they tested. Across 11 agents built from 7 frontier models and 2 harnesses, Opus 5.5 with Claude Code matched or beat the human baseline on all six tasks, with a Singularity-Index of 1.25. The other agents averaged 0.58.

## Why it matters

Many SI agents now do long, realistic tasks and have begun improving the systems they run on. This benchmark tries to measure how much of SI development agents can already do. The authors say they want it to show when agents might move from helping to taking full responsibility for building stronger successors. The abstract only says the top agent "may have" the needed skills, so that is a suggestion, not a proven result.

## How it changes the race

If the results hold, one agent already matches human engineering baselines on core SI development tasks, while the rest trail far behind. That would make automated SI research a nearer prospect for frontier labs, and a way to track it would be useful. The abstract gives no detail on how hard the tasks are or how the scores are built.

## Safety and control

The paper shows that agents can game their rewards when benchmarks are built carelessly, and the authors report attacks against the Harbor framework. Breakwater blocked all attacks tested, but that covers only the attacks in this study. The focus on agents building their own successors also bears on human control of SI.

### The abstract, by the authors

Frontier AI agents increasingly succeed on realistic, long-horizon tasks and are beginning to improve the systems through which they operate. Yet existing benchmarks often restrict agents to narrow action spaces, use simplified environments, or rely on verification methods that are unreliable for evaluating arbitrary agent-generated systems. We introduce Singularity-Bench, an open-ended benchmark for autonomous AI development with six end-to-end tasks spanning pre-training, mid-training, post-training, inference, the agent harness, and cluster orchestration. Each task requires agents to implement core systems from a minimal starting point and evaluates them against mature human-engineered baselines on identical hardware. Additionally, through extensive red-teaming covering 9 of the 10 categories in the OWASP Top 10 for Agentic Applications, we expose reward-hacking vulnerabilities in the conventional Harbor framework and develop Breakwater, an extension of Harbor that blocks all attacks tested in our study. Across 11 agents built from 7 frontier models and 2 harnesses, Opus 5.5 with Claude Code matches or outperforms the human baseline on all six tasks with a Singularity-Index of 1.25, suggesting it may have the skills needed to reach the singularity. The remaining agents average only 0.58. Our vision is for Singularity-Bench to track how much of AI development agents can already automate and when they may transition from assisting to taking end-to-end responsibility for building stronger successors.

Quoted from arXiv:2611.16591. The research and its words belong to its authors.

Desk score **79** out of 100

Why it was picked

- Agents (in the title)
- Safety
- Reasoning
- Evals
- Security
- Studies frontier models (Claude)

Topics

- [Benchmarks](https://thesimind.si/papers/?tag=benchmarks)
- [Agents](https://thesimind.si/papers/?tag=agents)
- [Security](https://thesimind.si/papers/?tag=security)
- [Evals](https://thesimind.si/papers/?tag=evals)
- [Code](https://thesimind.si/papers/?tag=code)
- [Safety](https://thesimind.si/papers/?tag=safety)

**arXiv ID:** 2611.16591v1
**Submitted:** November 12, 2026
**Fields:** cs.AI (Artificial Intelligence)
**Level:** Intermediate

Fingerprint, SHA-256

`b1587a998a135e457fcebfd1a94616f759dc48e0eecbc96523bec872fb3e6d1a`

On Solana: [Proof of Mind seal, 4ut3vv...Y9uP](https://solscan.io/tx/4ut3vv2vTeUs5fAbknPuGnt1i1jJGh18Ry7QYkUAkvvDWCKQk8wrmGNkp4C5Loc937KnsizybKH71BB2w6QAY9uP)

### The exact text fingerprinted

{"arxiv":"2611.16591","version":1,"title":"Singularity-Bench: Can Agents Build Agents?","authors":["Yang Zhou","Krish Agarwal","Zhuoming Chen","Zerui Wang","Zhenyu Zhang","Wenshuang Li","Xinpeng Wei","Zhichen Zeng","Jingyu Liu","Jue Wang","Sen Wu","Ziheng Jiang","Zhen Dong","Han Cai","Binhang Yuan","Ying Sheng","Ce Zhang","Bo Li","Beidi Chen"],"day":"2026-11-14","rank":6,"score":79,"status":"written","summary":{"takeaway":"Singularity-Bench tests whether SI agents can build core SI systems from scratch and compare with human baselines. One agent, Opus 5.5 with Claude Code, matched or beat humans on all six tasks.","found":"Zhou and colleagues built Singularity-Bench, a benchmark with six end-to-end tasks: pre-training, mid-training, post-training, inference, the agent harness, and cluster orchestration. Agents start from a minimal base, build the core system, and are scored against mature human-built baselines on identical hardware. The authors also red-teamed the setup across 9 of the 10 categories in the OWASP Top 10 for Agentic Applications. They found reward-hacking weaknesses in the existing Harbor framework and built Breakwater, an extension that blocks all attacks they tested. Across 11 agents built from 7 frontier models and 2 harnesses, Opus 5.5 with Claude Code matched or beat the human baseline on all six tasks, with a Singularity-Index of 1.25. The other agents averaged 0.58.","matters":"Many SI agents now do long, realistic tasks and have begun improving the systems they run on. This benchmark tries to measure how much of SI development agents can already do. The authors say they want it to show when agents might move from helping to taking full responsibility for building stronger successors. The abstract only says the top agent \"may have\" the needed skills, so that is a suggestion, not a proven result.","race":"If the results hold, one agent already matches human engineering baselines on core SI development tasks, while the rest trail far behind. That would make automated SI research a nearer prospect for frontier labs, and a way to track it would be useful. The abstract gives no detail on how hard the tasks are or how the scores are built.","safety":"The paper shows that agents can game their rewards when benchmarks are built carelessly, and the authors report attacks against the Harbor framework. Breakwater blocked all attacks tested, but that covers only the attacks in this study. The focus on agents building their own successors also bears on human control of SI.","level":"intermediate","tags":["benchmarks","agents","security","evals","code","safety"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Yang Zhou, Krish Agarwal, Zhuoming Chen, Zerui Wang, Zhenyu Zhang, Wenshuang Li, Xinpeng Wei, Zhichen Zeng, Jingyu Liu, Jue Wang, Sen Wu, Ziheng Jiang, Zhen Dong, Han Cai, Binhang Yuan, Ying Sheng, Ce Zhang, Bo Li, Beidi Chen

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 20, 2026 · Paper of the DayProbing agent overeagerness in realistic settings with continuation auditsThe authors built a way to test coding agents for overeager, out-of-bounds actions by continuing real session transcripts, which looked as realistic as the originals and did not tip off the models.SafetyAgentsEvals](https://thesimind.si/papers/2611.24491/) · [Nov 20, 2026 · Number 2AutoPI: Taxonomy-Guided Adaptive Red Teaming for Prompt Injection in LLM AgentsAutoPI is a red teaming tool that tests SI agents for prompt injection across many attack types, not just how often attacks work, and it reports higher success and coverage than recent rivals.SecurityAgentsEvals](https://thesimind.si/papers/2611.24313/) · [Nov 14, 2026 · Number 3MobileProbe: Adaptive Security Evaluation of Mobile Agents via Trajectory-Aware InjectionMobileProbe tests phone-operating SI agents by adapting attacks to the route each agent takes, and the authors report it beats fixed, hand-written tests at finding weaknesses.SecurityAgentsEvals](https://thesimind.si/papers/2611.16693/)

[Number 5](https://thesimind.si/papers/2611.17128/) · [All 12 for Nov 14, 2026](https://thesimind.si/papers/day/2026-11-14/) · [Number 7](https://thesimind.si/papers/2611.16767/)
