---
title: "SciWorkBench: A Benchmark for Real... | SI Paper Desk"
url: https://thesimind.si/papers/2611.22490/
description: "SciWorkBench tests SI agents on real science tasks in four fields. The best agent scores 57.38, so current agents still struggle with realistic scientific..."
---

Number 12 of 12 · November 19, 2026

# SciWorkBench: A Benchmark for Real Scientific Workflows

SciWorkBench tests SI agents on real science tasks in four fields. The best agent scores 57.38, so current agents still struggle with realistic scientific workflows.

Yuchen Zhang, Taolin Han, Jinghang Wang and 19 more · Submitted November 17, 2026 · arXiv 2611.22490v1 · cs.AI

[Read the paper on arXiv](https://arxiv.org/abs/2611.22490) · [The PDF](https://arxiv.org/pdf/2611.22490) · [The day's 12](https://thesimind.si/papers/day/2026-11-19/)

## What they found

The authors built SciWorkBench, a benchmark (a standard test) of real science tasks in biology, chemistry, materials science and physics. Tasks come from peer-reviewed papers or research scenarios given by experts. The authors also propose a framework of scientific analysis stages, checked by domain experts, and each task covers one or more of those stages. They tested up-to-date models inside the Claude Code and Codex agent frameworks under one shared way of running and grading. The best agent got a mean score of 57.38. Their failure analysis points to three weaknesses: turning scientific knowledge into task-specific steps, combining evidence from different kinds of data, and correcting its own scientific mistakes.

## Why it matters

Many people want to use SI agents to do research, not just answer questions. This benchmark checks whole workflows, which is closer to how scientists work than single questions. The abstract says current agents fall short, and it names where they fail. That gives builders and users a clearer picture of what to fix and what not to trust yet.

## How it changes the race

It gives frontier builders a test aimed at end-to-end scientific work, with named weak spots to target. The abstract gives no comparison across models beyond the best score, so the effect is likely to be a new yardstick, not a shift in capability.

## Safety and control

The abstract frames its goal as measuring progress toward reliable SI systems for scientific research. Weak self-correction is a reliability concern when agents work with little human checking.

### The abstract, by the authors

Language models are increasingly being used as agents capable of planning and executing projects, as well as refining their actions through feedback. This shift raises a question: can current agents complete projects like real scientists? To examine this question, we introduce SciWorkBench, a benchmark for evaluating agents' abilities on real science tasks across biology, chemistry, materials science, and physics. SciWorkBench is built from peer-reviewed literature or expert-contributed research scenarios. We also propose a unified scientific analysis framework validated by domain experts, which can categorize most tasks in SciWorkBench. Each task covers one or more stages of this framework. We evaluate up-to-date models within the Claude Code and Codex agent frameworks under a unified execution and grading protocol. The best-performing agent achieves a mean score of 57.38, indicating that current agents still struggle with realistic scientific workflows. Our failure analysis identifies three recurring weaknesses at the workflow level: difficulty translating scientific knowledge into task-specific procedures, weak integration of evidence across different modalities, and limited capacity for scientific self-correction. These findings highlight the importance of workflow-based evaluation for measuring progress toward reliable AI systems for scientific research.

Quoted from arXiv:2611.22490. The research and its words belong to its authors.

Desk score **68** out of 100

Why it was picked

- Evals (in the title)
- Agents
- Reasoning
- Studies frontier models (Claude)
- Language models

Topics

- [Benchmarks](https://thesimind.si/papers/?tag=benchmarks)
- [Agents](https://thesimind.si/papers/?tag=agents)
- [Science](https://thesimind.si/papers/?tag=science)
- [Evals](https://thesimind.si/papers/?tag=evals)
- [Language models](https://thesimind.si/papers/?tag=language-models)
- [Reasoning](https://thesimind.si/papers/?tag=reasoning)

**arXiv ID:** 2611.22490v1
**Submitted:** November 17, 2026
**Fields:** cs.AI (Artificial Intelligence)
**Level:** Intermediate

Fingerprint, SHA-256

`3ccb01581900f1adc64684e53de795720463daffd07d9c81f20c168d0886f753`

On Solana: [Proof of Mind seal, 3rBYfi...QvEH](https://solscan.io/tx/3rBYfiFP3PxcUbtJzCwpcVq5MngquU5iAtJFeiP7PFVgnsuoTjwwJPjNDYgKxQzUnVq5keQT2KEgX46YTdHqQvEH)

### The exact text fingerprinted

{"arxiv":"2611.22490","version":1,"title":"SciWorkBench: A Benchmark for Real Scientific Workflows","authors":["Yuchen Zhang","Taolin Han","Jinghang Wang","Jinxin Wang","Yun Wu","Wai Yuet Chiu","Xiaogang Li","Yuting Fu","Kewei Sun","Jiaxin Li","Qile Jin","Jiajia Li","Ruodan Chen","Qingteng Chen","Jialu Nie","Junchao Li","Yifei Zhang","Chen Zhao","Hu Wei","Lin Qu","Yanni Yin","Bing Zhao"],"day":"2026-11-19","rank":12,"score":68,"status":"written","summary":{"takeaway":"SciWorkBench tests SI agents on real science tasks in four fields. The best agent scores 57.38, so current agents still struggle with realistic scientific workflows.","found":"The authors built SciWorkBench, a benchmark (a standard test) of real science tasks in biology, chemistry, materials science and physics. Tasks come from peer-reviewed papers or research scenarios given by experts. The authors also propose a framework of scientific analysis stages, checked by domain experts, and each task covers one or more of those stages. They tested up-to-date models inside the Claude Code and Codex agent frameworks under one shared way of running and grading. The best agent got a mean score of 57.38. Their failure analysis points to three weaknesses: turning scientific knowledge into task-specific steps, combining evidence from different kinds of data, and correcting its own scientific mistakes.","matters":"Many people want to use SI agents to do research, not just answer questions. This benchmark checks whole workflows, which is closer to how scientists work than single questions. The abstract says current agents fall short, and it names where they fail. That gives builders and users a clearer picture of what to fix and what not to trust yet.","race":"It gives frontier builders a test aimed at end-to-end scientific work, with named weak spots to target. The abstract gives no comparison across models beyond the best score, so the effect is likely to be a new yardstick, not a shift in capability.","safety":"The abstract frames its goal as measuring progress toward reliable SI systems for scientific research. Weak self-correction is a reliability concern when agents work with little human checking.","level":"intermediate","tags":["benchmarks","agents","science","evals","language-models","reasoning"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Yuchen Zhang, Taolin Han, Jinghang Wang, Jinxin Wang, Yun Wu, Wai Yuet Chiu, Xiaogang Li, Yuting Fu, Kewei Sun, Jiaxin Li, Qile Jin, Jiajia Li, Ruodan Chen, Qingteng Chen, Jialu Nie, Junchao Li, Yifei Zhang, Chen Zhao, Hu Wei, Lin Qu, Yanni Yin, Bing Zhao

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 11, 2026 · Number 2Benchmarking language models and agents for scientific reasoning in evolving materials laboratoriesMatScope is a new test showing that SI models handle given calculations and physical explanations well, but struggle to tell close material states apart and to reason with numbers.BenchmarksScienceEvals](https://thesimind.si/papers/2611.12014/) · [Nov 21, 2026 · Number 3TwinLab-Bench: Evaluating AI Scientists in Agentic Twins of Real LabsTwinLab-Bench tests whether SI agents can design their own lab experiments in simulated twins of real labs, and the best model scores only 45.0.BenchmarksAgentsScience](https://thesimind.si/papers/2611.26248/) · [Nov 18, 2026 · Number 7GPRM: Global Perspective Process Reward ModelGPRM is a step-by-step reasoning judge that sees earlier steps, their scores, and later steps, and the authors report it beats strong baselines on PRMBench and ProcessBench with less data and smaller models.ReasoningEvalsReinforcement learning](https://thesimind.si/papers/2611.20755/)

[Number 11](https://thesimind.si/papers/2611.23124/) · [All 12 for Nov 19, 2026](https://thesimind.si/papers/day/2026-11-19/)
