---
title: "DECODE: Diagnosing Interpretability... | SI Paper Desk"
url: https://thesimind.si/papers/2611.17856/
description: "DECODE is a visual tool that puts humans back into automated SI interpretation, helping find where explanations of a model's inner features go wrong and why."
---

Number 7 of 12 · November 16, 2026

# DECODE: Diagnosing Interpretability Failures in Automated Interpretation Process

DECODE is a visual tool that puts humans back into automated SI interpretation, helping find where explanations of a model's inner features go wrong and why.

Dohyun Kim, Jaeung Lee, Donghee Hong and 1 more · Submitted November 13, 2026 · arXiv 2611.17856v1 · cs.AI

[Read the paper on arXiv](https://arxiv.org/abs/2611.17856) · [The PDF](https://arxiv.org/pdf/2611.17856) · [The day's 12](https://thesimind.si/papers/day/2026-11-16/)

## What they found

The authors build DECODE, a visual analytics system (an interactive tool with charts and views) for checking automated interpretation of LLMs. In these pipelines, an LLM both writes and scores explanations of what a model's internal parts do, so there is little human oversight and no ground truth. DECODE splits diagnosis into three stages, so failures can be traced from structural defects to explanation-level misalignments. It treats agreement among independently generated explanations and their evaluation metrics as a stand-in for reliability, an idea borrowed from inter-coder consensus in qualitative research. A three-phase active seeding protocol recommends which items humans should review first. In a case study on 16,384 Sparse Autoencoder features, the authors say it can efficiently find and attribute interpretability failures at scale. The abstract gives no numbers on accuracy or on how it compares with other methods.

## Why it matters

Interpretability tools are meant to help people understand what is happening inside SI models. If the explanations come from another LLM and are scored by an LLM, errors can hide inside the loop. This work tackles a practical question: when an explanation looks wrong, is the fault in the tool's extracted features or in the LLM that explained them? Better ways to answer that make interpretability results easier to trust.

## How it changes the race

This is a supporting tool, not a capability advance. If it holds up, it could make large-scale interpretability work more reliable and cut the human effort needed to audit it.

## Safety and control

The paper bears on human oversight of SI. It aims to make automated interpretation checkable by people, which matters if interpretability is used to audit models. The abstract only reports one case study, so how well it works more broadly is not shown.

### The abstract, by the authors

Tools for interpreting the internal representations of Large Language Models (LLMs) increasingly rely on LLMs themselves to both generate and evaluate explanations, creating a self-referential automated interpretation process with minimal human oversight. This automation raises significant reliability concerns, which are exacerbated by the absence of ground truth. In practice, it is difficult to validate explanations or to attribute interpretability failures to their source - whether they originate from the tool's extracted representations or from the LLM used in the process. To address these challenges, we present DECODE, a visual analytics system that reintroduces human oversight into the automated interpretation process. Our approach decomposes the diagnostic workflow into three progressive stages, isolating failure sources ranging from structural defects to explanation-level misalignments. Drawing inspiration from inter-coder consensus in qualitative analysis, we utilize consensus among independently generated explanations and their evaluation metrics as a proxy for reliability. To further scale human inspection, we employ a three-phase active seeding protocol that strategically recommends high-priority instances for manual review. Finally, we demonstrate the effectiveness of our approach through a case study on 16,384 Sparse Autoencoder features, showing that it can efficiently identify and attribute interpretability failures at scale.

Quoted from arXiv:2611.17856. The research and its words belong to its authors.

Desk score **77** out of 100

Why it was picked

- Interpretability (in the title)
- Safety
- Alignment
- Evals
- Language models
- Cross-listed in 2 fields

Topics

- [Interpretability](https://thesimind.si/papers/?tag=interpretability)
- [Evals](https://thesimind.si/papers/?tag=evals)
- [Language models](https://thesimind.si/papers/?tag=language-models)
- [Safety](https://thesimind.si/papers/?tag=safety)
- [Alignment](https://thesimind.si/papers/?tag=alignment)

**arXiv ID:** 2611.17856v1
**Submitted:** November 13, 2026
**Fields:** cs.AI (Artificial Intelligence), cs.HC (Human-Computer Interaction)
**Level:** Intermediate

Fingerprint, SHA-256

`32e8fbef50e6a71184a13071956835666c617c9cd107657f1348f79a9e0b837f`

On Solana: [Proof of Mind seal, 3mXUMW...GSim](https://solscan.io/tx/3mXUMW5MkdFiyPBNPYd5DfwHhma5EdjhUFEw9jZaDuuPqgszJtoDYttSrD9kyWUFdMCzn7RnFNd4nQhwAUgrGSim)

### The exact text fingerprinted

{"arxiv":"2611.17856","version":1,"title":"DECODE: Diagnosing Interpretability Failures in Automated Interpretation Process","authors":["Dohyun Kim","Jaeung Lee","Donghee Hong","Jaemin Jo"],"day":"2026-11-16","rank":7,"score":77,"status":"written","summary":{"takeaway":"DECODE is a visual tool that puts humans back into automated SI interpretation, helping find where explanations of a model's inner features go wrong and why.","found":"The authors build DECODE, a visual analytics system (an interactive tool with charts and views) for checking automated interpretation of LLMs. In these pipelines, an LLM both writes and scores explanations of what a model's internal parts do, so there is little human oversight and no ground truth. DECODE splits diagnosis into three stages, so failures can be traced from structural defects to explanation-level misalignments. It treats agreement among independently generated explanations and their evaluation metrics as a stand-in for reliability, an idea borrowed from inter-coder consensus in qualitative research. A three-phase active seeding protocol recommends which items humans should review first. In a case study on 16,384 Sparse Autoencoder features, the authors say it can efficiently find and attribute interpretability failures at scale. The abstract gives no numbers on accuracy or on how it compares with other methods.","matters":"Interpretability tools are meant to help people understand what is happening inside SI models. If the explanations come from another LLM and are scored by an LLM, errors can hide inside the loop. This work tackles a practical question: when an explanation looks wrong, is the fault in the tool's extracted features or in the LLM that explained them? Better ways to answer that make interpretability results easier to trust.","race":"This is a supporting tool, not a capability advance. If it holds up, it could make large-scale interpretability work more reliable and cut the human effort needed to audit it.","safety":"The paper bears on human oversight of SI. It aims to make automated interpretation checkable by people, which matters if interpretability is used to audit models. The abstract only reports one case study, so how well it works more broadly is not shown.","level":"intermediate","tags":["interpretability","evals","language-models","safety","alignment"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Dohyun Kim, Jaeung Lee, Donghee Hong, Jaemin Jo

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 18, 2026 · Number 3Abstention Direction in Large Language Models: Mechanistic Interpretability of Abstention Across Query Ambiguity, Knowledge Limits, and Safety AlignmentThe authors find directions inside language models that control abstention (declining to answer) for unclear questions, unknown facts and harmful requests, and these directions partly overlap.InterpretabilityAlignmentSafety](https://thesimind.si/papers/2611.20441/) · [Nov 9, 2026 · Number 2Forcing a Letter Suppresses Refusal: Mechanistic Evidence from WMDPIn multiple-choice safety tests, forcing a model to pick a letter seems to crowd out its refusal, so these scores may measure knowledge more than safety.SafetyInterpretabilityEvals](https://thesimind.si/papers/2611.08639/) · [Nov 12, 2026 · Number 8TERM: Target-Free Jailbreak Defense via Entropy-Guided Residual ModulationTERM is a defense that runs while a model answers. It watches how the model's attention shifts from its own starting point and scales down the model's internal updates when something looks off.SecuritySafetyAlignment](https://thesimind.si/papers/2611.13577/)

[Number 6](https://thesimind.si/papers/2611.17398/) · [All 12 for Nov 16, 2026](https://thesimind.si/papers/day/2026-11-16/) · [Number 8](https://thesimind.si/papers/2611.17620/)
