Number 12 of 12 · November 19, 2026

SciWorkBench: A Benchmark for Real Scientific Workflows

SciWorkBench tests SI agents on real science tasks in four fields. The best agent scores 57.38, so current agents still struggle with realistic scientific workflows.

What they found

The authors built SciWorkBench, a benchmark (a standard test) of real science tasks in biology, chemistry, materials science and physics. Tasks come from peer-reviewed papers or research scenarios given by experts. The authors also propose a framework of scientific analysis stages, checked by domain experts, and each task covers one or more of those stages. They tested up-to-date models inside the Claude Code and Codex agent frameworks under one shared way of running and grading. The best agent got a mean score of 57.38. Their failure analysis points to three weaknesses: turning scientific knowledge into task-specific steps, combining evidence from different kinds of data, and correcting its own scientific mistakes.

Why it matters

Many people want to use SI agents to do research, not just answer questions. This benchmark checks whole workflows, which is closer to how scientists work than single questions. The abstract says current agents fall short, and it names where they fail. That gives builders and users a clearer picture of what to fix and what not to trust yet.

How it changes the race

It gives frontier builders a test aimed at end-to-end scientific work, with named weak spots to target. The abstract gives no comparison across models beyond the best score, so the effect is likely to be a new yardstick, not a shift in capability.

Safety and control

The abstract frames its goal as measuring progress toward reliable SI systems for scientific research. Weak self-correction is a reliability concern when agents work with little human checking.

The abstract, by the authors

Language models are increasingly being used as agents capable of planning and executing projects, as well as refining their actions through feedback. This shift raises a question: can current agents complete projects like real scientists? To examine this question, we introduce SciWorkBench, a benchmark for evaluating agents' abilities on real science tasks across biology, chemistry, materials science, and physics. SciWorkBench is built from peer-reviewed literature or expert-contributed research scenarios. We also propose a unified scientific analysis framework validated by domain experts, which can categorize most tasks in SciWorkBench. Each task covers one or more stages of this framework. We evaluate up-to-date models within the Claude Code and Codex agent frameworks under a unified execution and grading protocol. The best-performing agent achieves a mean score of 57.38, indicating that current agents still struggle with realistic scientific workflows. Our failure analysis identifies three recurring weaknesses at the workflow level: difficulty translating scientific knowledge into task-specific procedures, weak integration of evidence across different modalities, and limited capacity for scientific self-correction. These findings highlight the importance of workflow-based evaluation for measuring progress toward reliable AI systems for scientific research.

Quoted from arXiv:2611.22490. The research and its words belong to its authors.

The authors

Yuchen Zhang, Taolin Han, Jinghang Wang, Jinxin Wang, Yun Wu, Wai Yuet Chiu, Xiaogang Li, Yuting Fu, Kewei Sun, Jiaxin Li, Qile Jin, Jiajia Li, Ruodan Chen, Qingteng Chen, Jialu Nie, Junchao Li, Yifei Zhang, Chen Zhao, Hu Wei, Lin Qu, Yanni Yin, Bing Zhao

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.