What they found
The authors built DeepResearch RewardBench, which they call the first benchmark for reward modeling on research reports. A reward model is a system that scores or ranks outputs. Reports come from multiple agents or tool-augmented LLMs and are scored with query-specific golden rubrics. The authors used multi-round filtering and human verification to keep quality and difficulty up. The data comes as pairs, triplets and quadruplets, so the test covers ranking at different difficulty levels. In their experiments, current reward models handle general coherence well but get much worse on research reports. Generative LLMs used as judges did better overall, and comparing reports against each other worked better than scoring each one directly. The abstract gives no specific scores.
Why it matters
Deep research systems write long, dense reports that are costly for humans to check. Teams therefore need SI judges they can trust to score them. This paper gives a way to measure whether those judges are reliable. It also points to a gap: reward models that work on general text may not transfer to this task.
How it changes the race
If the results hold, builders of research agents get a shared test for their judging models, and a hint that comparative LLM judging is a stronger base than direct scoring. This is a measurement tool, so it supports progress more than it changes capability directly.
Safety and control
Reward models guide how SI systems are trained and checked. This benchmark shows they can be weak on long, specialized reports, which is a reason to be careful about relying on them without human checks.
The abstract, by the authors
Empowered by Large Language Models (LLMs), deep research systems integrate online information into research reports through multi-step web searching and information synthesis. Evaluating these reports presents a significant challenge due to their long context, high information density, and domain specificity, necessitating the use of AI evaluation as a substitute for prohibitively expensive human assessment. However, few studies have examined the effectiveness of existing reward models or LLMs in understanding and evaluating such research reports. To bridge this gap, we introduce DeepResearch RewardBench, the first benchmark dedicated to evaluating reward modeling for research reports. We collect diverse report candidates from multiple agents or tool-augmented LLMs. These candidates are scored using query-specific golden rubrics, and we ensure data quality and difficulty through a combination of multi-round filtering and human verification. Our benchmark comprises preference datasets formatted as pairs, triplets, and quadruplets to test ranking capabilities across varying levels of difficulty. Extensive experiments demonstrate that while current reward models handle general coherence effectively, their performance significantly degrades when evaluating research reports. Generative LLMs acting as judges demonstrate comprehensive superiority in this domain, with comparative evaluation proving more effective than direct scoring methods. We anticipate that this benchmark will serve as a vital testbed for the evaluation and advancement of research agents.
Quoted from arXiv:2611.10156. The research and its words belong to its authors.