Number 11 of 12 · November 16, 2026

Tracing Knowledge and Reasoning in LLM

Yang Wang and colleagues propose a way to look inside language models while they take benchmarks, to show whether a task needs recall of facts or real reasoning, not just a score.

What they found

The authors build a framework that looks at what happens inside a model when it works through a benchmark. It combines attention signals, activation signals and measures of internal representations. The goal is to place benchmarks on a range from retrieval-dominant (recalling facts) to reasoning-dominant, and to reveal failure modes that a single score hides. They test 16 LLMs on 24 diverse benchmarks and run deeper internal-state checks on chosen models and tasks. They report systematic differences in processing between knowledge-centric and reasoning-centric tasks. The abstract gives no specific numbers on how large these differences are.

Why it matters

Most model tests report one score per task. That shows what a model achieves but not how it got there. This work asks what kind of processing sits behind a score. If it holds, people could choose benchmarks more carefully and pick models with more insight into what each one is good at. The authors describe these benefits as potential, not proven. The code is open source.

How it changes the race

This is mainly an evaluation tool, so it is incremental for capability. It could help labs and users judge which benchmarks truly test reasoning and which mostly test recall.

Safety and control

By surfacing failure modes that task-level scores conceal, this kind of analysis could help people understand SI model behavior better. The abstract does not make direct safety claims.

The abstract, by the authors

Current evaluations of Large Language Models (LLMs) report performance at the dataset or task level, revealing what a model achieves but offering little insight into the internal processing associated with its performance. We propose a task-centric framework for characterising the model-internal processing elicited by a benchmark. It combines attention- and activation-based signals with representation-based measures to characterise benchmarks spanning a continuum from retrieval-dominant to reasoning-dominant and to surface failure modes that task-level scores conceal. We evaluate 16 LLMs across 24 diverse benchmarks and conduct targeted internal-state analyses on representative models and tasks, observing systematic differences in processing between knowledge-centric and reasoning-centric tasks. We shift evaluation from asking what score a model achieves toward characterising the processing patterns associated with that score, potentially supporting more targeted benchmark selection and more informed model choice. We open source our code at https://github.com/extraordinarylab/trace-kr.

Quoted from arXiv:2611.17461. The research and its words belong to its authors.

The authors

Yang Wang, Xiao Wang, Yiqi Liu, Chenghao Xiao, Eleni Vasilaki, Goran Nenadic, Chenghua Lin

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.