---
title: "AnalogLayoutWorld: An Open Benchmark for... | SI Paper Desk"
url: https://thesimind.si/papers/2611.16991/
description: "SI coding agents can often produce analog chip layout files, but only 23.0% of attempts pass the benchmark's acceptance criteria, so physical engineering..."
---

Number 5 of 12 · November 15, 2026

# AnalogLayoutWorld: An Open Benchmark for Agentic Analog Layout Design

SI coding agents can often produce analog chip layout files, but only 23.0% of attempts pass the benchmark's acceptance criteria, so physical engineering remains hard.

NovaSilicon Team · Submitted November 12, 2026 · arXiv 2611.16991v1 · cs.AI

[Read the paper on arXiv](https://arxiv.org/abs/2611.16991) · [The PDF](https://arxiv.org/pdf/2611.16991) · [The day's 12](https://thesimind.si/papers/day/2026-11-15/)

## What they found

The NovaSilicon Team built AnalogLayoutWorld, an open-source benchmark with 40 expert-reviewed tasks. Each task asks an SI agent to produce a chip layout from a fixed transistor-level netlist (a list of parts and how they connect). The benchmark gives agents a design environment and checks results with independent physical and post-layout electrical verification. It also uses layout quality metrics made with analog layout experts. Across 13 model-harness configurations, 91.1% of attempts produced GDS layouts (a standard chip layout file), but only 23.0% met the acceptance criteria. The authors also test Novara, their own internal agent system, which got the highest reported aggregate score. They link performance gaps to test-time scaling under fixed budgets, and suggest that reasoning efficiency and good use of verification feedback help quality hold up as tasks get harder.

## Why it matters

Coding agents do well at software, but real engineering work follows physical rules and needs many steps with tools. This benchmark gives a way to measure that gap. The large drop from producing a file to producing a correct layout shows that looking finished is not the same as being right. Independent verification is what exposes it.

## How it changes the race

It gives frontier SI builders an open test for long, physics-bound engineering work, beyond software tasks. The authors' own system leads the table, so that comparison deserves care until others run it.

## Safety and control

The gap between 91.1% of attempts producing layouts and 23.0% passing acceptance shows why independent checks matter before trusting SI output in engineering work.

### The abstract, by the authors

Coding agents have demonstrated strong capabilities in software development, yet their ability to tackle long-horizon engineering tasks governed by physical constraints remains less understood. We study this question through analog circuit layout, which demands domain knowledge, geometric reasoning, and iterative interaction with design and verification tools. We introduce AnalogLayoutWorld, an open-source agent benchmark with 40 expert-reviewed tasks for end-to-end layout generation from fixed transistor-level netlists. It combines an integrated design environment with independent physical and post-layout electrical verification, as well as objective layout quality metrics co-designed with analog layout experts. Across 13 model-harness configurations, 91.1% of attempts produce GDS layouts, but only 23.0% satisfy the benchmark's acceptance criteria. We also evaluate Novara, an internally developed agent system, which achieves the highest reported aggregate score. Our analysis links performance differences to test-time scaling under fixed budgets, suggesting that reasoning efficiency and effective use of verification feedback help sustain design quality as task complexity grows.

Quoted from arXiv:2611.16991. The research and its words belong to its authors.

Desk score **75** out of 100

Why it was picked

- Agents (in the title)
- Evals (in the title)
- Interpretability
- Reasoning

Topics

- [Benchmarks](https://thesimind.si/papers/?tag=benchmarks)
- [Agents](https://thesimind.si/papers/?tag=agents)
- [Evals](https://thesimind.si/papers/?tag=evals)
- [Reasoning](https://thesimind.si/papers/?tag=reasoning)
- [Code](https://thesimind.si/papers/?tag=code)
- [Science](https://thesimind.si/papers/?tag=science)

**arXiv ID:** 2611.16991v1
**Submitted:** November 12, 2026
**Fields:** cs.AI (Artificial Intelligence)
**Level:** Intermediate

Fingerprint, SHA-256

`05ce95cf172b2b4870762eb85dd8eac4799875a8b2cda4414b915cb86b480e86`

On Solana: [Proof of Mind seal, Szprsf...FGjr](https://solscan.io/tx/SzprsfAMG4vXvYD7hrurYFce76smMcrTbLnANy8xS96bh14DoynCdjEEMZf7RPtRUgYeejS8HGnxjqtD1XuFGjr)

### The exact text fingerprinted

{"arxiv":"2611.16991","version":1,"title":"AnalogLayoutWorld: An Open Benchmark for Agentic Analog Layout Design","authors":["NovaSilicon Team"],"day":"2026-11-15","rank":5,"score":75,"status":"written","summary":{"takeaway":"SI coding agents can often produce analog chip layout files, but only 23.0% of attempts pass the benchmark's acceptance criteria, so physical engineering remains hard.","found":"The NovaSilicon Team built AnalogLayoutWorld, an open-source benchmark with 40 expert-reviewed tasks. Each task asks an SI agent to produce a chip layout from a fixed transistor-level netlist (a list of parts and how they connect). The benchmark gives agents a design environment and checks results with independent physical and post-layout electrical verification. It also uses layout quality metrics made with analog layout experts. Across 13 model-harness configurations, 91.1% of attempts produced GDS layouts (a standard chip layout file), but only 23.0% met the acceptance criteria. The authors also test Novara, their own internal agent system, which got the highest reported aggregate score. They link performance gaps to test-time scaling under fixed budgets, and suggest that reasoning efficiency and good use of verification feedback help quality hold up as tasks get harder.","matters":"Coding agents do well at software, but real engineering work follows physical rules and needs many steps with tools. This benchmark gives a way to measure that gap. The large drop from producing a file to producing a correct layout shows that looking finished is not the same as being right. Independent verification is what exposes it.","race":"It gives frontier SI builders an open test for long, physics-bound engineering work, beyond software tasks. The authors' own system leads the table, so that comparison deserves care until others run it.","safety":"The gap between 91.1% of attempts producing layouts and 23.0% passing acceptance shows why independent checks matter before trusting SI output in engineering work.","level":"intermediate","tags":["benchmarks","agents","evals","reasoning","code","science"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

NovaSilicon Team

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 11, 2026 · Number 2Benchmarking language models and agents for scientific reasoning in evolving materials laboratoriesMatScope is a new test showing that SI models handle given calculations and physical explanations well, but struggle to tell close material states apart and to reason with numbers.BenchmarksScienceEvals](https://thesimind.si/papers/2611.12014/) · [Nov 11, 2026 · Number 3Publications as benchmarks: auditing AI agents on spatial transcriptomics alignmentThe authors turn published papers into auditable tests for SI agents, and on spatial transcriptomics alignment the best agents matched or beat all baseline methods on 80% of 121 tasks.EvalsBenchmarksAgents](https://thesimind.si/papers/2611.11796/) · [Nov 11, 2026 · Number 8Late but Valuable: Why Deadlines Change the Measured Value of LLM ReasoningWhen SI answers must arrive before a deadline, judging answer quality and speed separately overstates the value of reasoning. Score the action that actually runs.EvalsAgentsReasoning](https://thesimind.si/papers/2611.11910/)

[Number 4](https://thesimind.si/papers/2611.16433/) · [All 12 for Nov 15, 2026](https://thesimind.si/papers/day/2026-11-15/) · [Number 6](https://thesimind.si/papers/2611.17167/)
