What they found
The authors studied one failure: web agents that try actions a website does not offer. They ran several interpretability analyses on Qwen2.5-7B-Instruct while it used live websites. A simple linear probe (a small classifier) on the last prompt token predicted hallucinations before generation began, with 0.80 AUC. Steering the model along the probe direction corrected 85% of hallucinations but disrupted 7% of valid actions. The authors say this is only a partial explanation. The probe direction was sensitive to prompt formatting and lost much of its steering effect after a minor change to how actions are shown. Circuit finding was hard because the computation is spread across many components and positions, and the information the agent uses differs from example to example.
Why it matters
Interpretability aims to explain why SI systems do what they do. This paper tests that aim on a realistic agent failure instead of a clean lab task. It shows where standard methods for finding circuits and features run into trouble. The authors outline future directions for making these methods better suited to agent failures. The abstract reports one model and one failure type, so how far the findings carry over is not stated.
How it changes the race
This is a case study and does not change capability or cost for frontier builders. It suggests that interpretability tools may need rework before they can explain the failures of SI agents in real settings.
Safety and control
The paper bears on human understanding and control of SI agents. A probe that flags likely errors before they happen is a useful signal. But the fragile steering and the partial explanation suggest these tools should not yet be trusted as reliable safeguards.
The abstract, by the authors
Mechanistic Interpretability (MI) has made significant progress in explaining the internal computations of Large Language Models (LLMs); however, these advances have yet to demonstrate comparable success in explaining the decision-making processes of LLM-based agents. This paper examines the challenges MI tools face when applied to realistic agentic failures. We focus on one such failure: web agents hallucinating actions that are not available on a website. We apply a series of mechanistic analyzes to Qwen2.5-7B-Instruct as it interacts with live websites. We find that a linear probe trained on the last prompt token can predict hallucinations before generation begins (0.80 AUC) and steering along a probe direction corrects 85% of hallucinations, but disrupts 7% of valid actions. However, these results only provide a partial explanation of why the agent hallucinates. The probe direction is sensitive to prompt formatting, losing much of its steering effect after a minor change to how actions are represented. Additionally, circuit identification can be difficult as the computation is distributed across multiple components and positions, and the information the agent relies on varies across examples. We discuss how these observations challenge the assumptions underlying standard MI methods for circuit discovery and feature identification, and outline future directions for making these methods better suited to explain failures in LLM-based agents.
Quoted from arXiv:2611.18514. The research and its words belong to its authors.
