---
title: "More Agents, More Samples, or More... | SI Paper Desk"
url: https://thesimind.si/papers/2611.17610/
description: "When multi-agent SI systems are compared with fair baselines that use the same number of samples or the same compute, they usually do no better and often do..."
---

Number 3 of 12 · November 16, 2026

# More Agents, More Samples, or More Thinking? Untangling Coordination, Ensembling, and Test-Time Compute in Multi-Agent LLMs

When multi-agent SI systems are compared with fair baselines that use the same number of samples or the same compute, they usually do no better and often do slightly worse.

Seyed Mahmoud Sajjadi Mohammadabadi · Submitted November 13, 2026 · arXiv 2611.17610v1 · cs.AI

[Read the paper on arXiv](https://arxiv.org/abs/2611.17610) · [The PDF](https://arxiv.org/pdf/2611.17610) · [The day's 12](https://thesimind.si/papers/day/2026-11-16/)

## What they found

The author pulls together nine earlier empirical studies, covering 516 multi-agent setups. Many of those studies compared agents that talk to each other with a single greedy pass, which is one quick answer from one model. That mixes up three things: coordination, the number of samples, and the amount of compute. Against a single-pass baseline, multi-agent accuracy differences are negative overall. They are near zero if the largest study is left out. Against baselines matched on sample count or resource budget, multi-agent systems average -0.042 in accuracy. They equal or fall below the baseline in 74% of these comparable contrasts. The author argues that communication adds no new information when agents share the same evidence. Gains need split-up evidence, outside verification, or varied errors across different models.

## Why it matters

Many claims that teams of SI agents beat one model may come from unfair comparisons. Part of the apparent gain may just come from drawing more samples or spending more compute. This paper gives builders and reviewers a way to check this. It offers a reviewer checklist and accounting tools to track resources. The abstract does not say how the nine studies were chosen, so the findings rest on that set.

## How it changes the race

If the result holds, teams may get more from spending compute on sampling or longer thinking by a single model than from complex multi-agent designs. Multi-agent setups would then be worth the cost only when agents hold different evidence, use external checks, or come from different models.

## Safety and control

The paper does not address safety directly. It does suggest that claims about agent teams need fairer testing before people trust them.

### The abstract, by the authors

Evaluations of multi-agent language models often compare communicating architectures against a single greedy pass. This practice confounds coordination with sample multiplicity and inference compute. We synthesize nine primary empirical evaluations comprising 516 unique multi-agent configurations organized along a resource vector R with six dimensions and a baseline ladder with four categories. Against baselines with a single pass (Category A), multi-agent accuracy differences are negative overall and near zero when omitting the largest contributing study. When evaluated against baselines matched on sample count (Category B) or resource budget (Category C), multi-agent systems average -0.042 in accuracy, and multilevel random effects modeling on the log odds scale remains negative across sensitivity checks. Multi-agent accuracy equals or falls below the baseline in 74% of these comparable contrasts, and study means are nonpositive in almost all evaluated studies. Across studies reporting both baselines, moving from a greedy pass to a sampled baseline shifts accuracy differences by -0.038 on average. Because communication adds no information when agents share identical evidence, positive returns require partitioned evidence, external verification, or error diversity across models. We provide a reviewer checklist and accounting utilities.

Quoted from arXiv:2611.17610. The research and its words belong to its authors.

Desk score **84** out of 100

Why it was picked

- Agents (in the title)
- Reasoning (in the title)
- Evals
- Multi-agent (in the title)
- Language models (in the title)
- Cross-listed in 4 fields
- Code or data released

Topics

- [Multi-agent systems](https://thesimind.si/papers/?tag=multi-agent)
- [Evals](https://thesimind.si/papers/?tag=evals)
- [Benchmarks](https://thesimind.si/papers/?tag=benchmarks)
- [Reasoning](https://thesimind.si/papers/?tag=reasoning)
- [Language models](https://thesimind.si/papers/?tag=language-models)
- [Compute](https://thesimind.si/papers/?tag=compute)

**arXiv ID:** 2611.17610v1
**Submitted:** November 13, 2026
**Fields:** cs.AI (Artificial Intelligence), cs.CL (Computation and Language), cs.LG (Machine Learning), cs.MA (Multiagent Systems)
**Level:** Intermediate
**Comments:** 43 pages, 11 figures, 13 tables. PACE Reviewer Checklist, evaluation utilities (pace CLI), and reproduction data at [https://github.com/mahmoudsajjadi/pace](https://github.com/mahmoudsajjadi/pace)

Fingerprint, SHA-256

`4db7f2c801b6f02e788cf353e3326d53a2cd2160e77176efbaef4509ebdf2625`

On Solana: [Proof of Mind seal, 3mXUMW...GSim](https://solscan.io/tx/3mXUMW5MkdFiyPBNPYd5DfwHhma5EdjhUFEw9jZaDuuPqgszJtoDYttSrD9kyWUFdMCzn7RnFNd4nQhwAUgrGSim)

### The exact text fingerprinted

{"arxiv":"2611.17610","version":1,"title":"More Agents, More Samples, or More Thinking? Untangling Coordination, Ensembling, and Test-Time Compute in Multi-Agent LLMs","authors":["Seyed Mahmoud Sajjadi Mohammadabadi"],"day":"2026-11-16","rank":3,"score":84,"status":"written","summary":{"takeaway":"When multi-agent SI systems are compared with fair baselines that use the same number of samples or the same compute, they usually do no better and often do slightly worse.","found":"The author pulls together nine earlier empirical studies, covering 516 multi-agent setups. Many of those studies compared agents that talk to each other with a single greedy pass, which is one quick answer from one model. That mixes up three things: coordination, the number of samples, and the amount of compute. Against a single-pass baseline, multi-agent accuracy differences are negative overall. They are near zero if the largest study is left out. Against baselines matched on sample count or resource budget, multi-agent systems average -0.042 in accuracy. They equal or fall below the baseline in 74% of these comparable contrasts. The author argues that communication adds no new information when agents share the same evidence. Gains need split-up evidence, outside verification, or varied errors across different models.","matters":"Many claims that teams of SI agents beat one model may come from unfair comparisons. Part of the apparent gain may just come from drawing more samples or spending more compute. This paper gives builders and reviewers a way to check this. It offers a reviewer checklist and accounting tools to track resources. The abstract does not say how the nine studies were chosen, so the findings rest on that set.","race":"If the result holds, teams may get more from spending compute on sampling or longer thinking by a single model than from complex multi-agent designs. Multi-agent setups would then be worth the cost only when agents hold different evidence, use external checks, or come from different models.","safety":"The paper does not address safety directly. It does suggest that claims about agent teams need fairer testing before people trust them.","level":"intermediate","tags":["multi-agent","evals","benchmarks","reasoning","language-models","compute"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Seyed Mahmoud Sajjadi Mohammadabadi

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 9, 2026 · Paper of the DayPhantom Consensus: Measuring Social Reasoning in Multi-Agent LLM Decision-MakingWhen SI models see ambiguous public signals from other models, they can drift to a group answer that no one privately held, and checking private judgments can catch this.Multi-agent systemsEvalsBenchmarks](https://thesimind.si/papers/2611.08971/) · [Nov 12, 2026 · Number 4Evaluating LLM-Generated World Models for Neuro-Symbolic ReasoningSI models can write world models that run without errors, but those models often differ a lot in structure from a human-made reference, so "it runs" does not mean "it is right."EvalsBenchmarksLanguage models](https://thesimind.si/papers/2611.14419/) · [Nov 11, 2026 · Number 2Benchmarking language models and agents for scientific reasoning in evolving materials laboratoriesMatScope is a new test showing that SI models handle given calculations and physical explanations well, but struggle to tell close material states apart and to reason with numbers.BenchmarksScienceEvals](https://thesimind.si/papers/2611.12014/)

[Number 2](https://thesimind.si/papers/2611.17880/) · [All 12 for Nov 16, 2026](https://thesimind.si/papers/day/2026-11-16/) · [Number 4](https://thesimind.si/papers/2611.17780/)
