---
title: "When can an interpretability claim be... | SI Paper Desk"
url: https://thesimind.si/papers/2611.16190/
description: "Claims that a sparse dictionary has recovered a model's features can flip with small changes, and the authors give provable stability bounds and..."
---

Number 9 of 12 · November 15, 2026

# When can an interpretability claim be trusted? Stability and certification of feature dimensionality, ridge leverage and the feature recovery rate

Claims that a sparse dictionary has recovered a model's features can flip with small changes, and the authors give provable stability bounds and certificates, tested only in toy models.

Ali Tajfar and Behnam Tajfar · Submitted November 12, 2026 · arXiv 2611.16190v1 · cs.LG

[Read the paper on arXiv](https://arxiv.org/abs/2611.16190) · [The PDF](https://arxiv.org/pdf/2611.16190) · [The day's 12](https://thesimind.si/papers/day/2026-11-15/)

## What they found

The authors study two common interpretability measurements in toy models and synthetic dictionaries where the true answer is known: feature dimensionality and the feature recovery rate (FRR) with a cosine threshold. They show exactly where feature dimensionality is discontinuous, meaning it can jump with tiny changes. They replace it with a ridge (regularized) version that has sharp stability bounds. The ridge leverage score changes by at most 3sqrt3/(8sqrtλ) times the operator-norm change of the weights, and this constant cannot be improved. They also prove certificates that say when a claim "feature i is recovered" must survive a given change of the dictionary. The certificates held in every test and are informative for small perturbations. They certify no feature non-trivially across independently trained dictionaries. At the standard threshold, whether a feature counted as recovered changed with the training seed for 55 percent of features.

## Why it matters

Interpretability results are used to support safety claims, such as "this dictionary found most of the model's features." This paper shows that such claims can depend on arbitrary thresholds, training seeds and small weight changes. It gives a way to say when a claim is stable and when it is not. That helps readers judge how much weight such claims deserve.

## How it changes the race

This is a methods and rigor paper, not a capability result. If it holds beyond toy models, teams building frontier SI would need to report stability checks with their interpretability claims, and some current claims may look weaker.

## Safety and control

The paper bears directly on safety evidence. It warns that interpretability-based safety claims, like feature recovery rates, can reverse under a different seed or threshold. The experiments use toy models only, so the abstract does not show how this applies to large SI systems. The authors also state that SI agents drafted the proofs, code and text, with an independent blind SI reviewer and numerical tests as checks.

### The abstract, by the authors

Interpretability tools increasingly support safety claims, such as that a sparse dictionary has recovered most of a model's features. Such claims rest on measurements with arbitrary thresholds, and small changes to the model weights, a different training seed or a different threshold can reverse them. We study two widely used measurements in toy models and synthetic dictionaries with known ground truth: the feature dimensionality of Elhage et al. and the feature recovery rate (FRR) with a cosine threshold. We characterize exactly where feature dimensionality is discontinuous and replace it with a regularized (ridge) version with sharp stability bounds: the ridge leverage score changes by at most 3sqrt3/(8sqrtλ) times the operator-norm change of the weights, and this constant cannot be improved; for the regularized dimensionality the stability constant grows like ρ/4 in the scale ρ=R/sqrtλ, with explicit upper and lower bounds. We also prove certificates stating when a claim "feature i is recovered" must survive a given change of the dictionary. The certificates held in every test and are informative for small perturbations, but certify no feature non-trivially across independently trained dictionaries; at the standard threshold, whether a feature counts as recovered changed with the training seed for 55 percent of features. Experiments use toy models only. The proofs, code and text were drafted by AI agents directed by the authors and checked by an independent blind AI reviewer and by numerical tests; the authors take full responsibility for the content.

Quoted from arXiv:2611.16190. The research and its words belong to its authors.

Desk score **67** out of 100

Why it was picked

- Interpretability (in the title)
- Safety
- Agents
- Cross-listed in 2 fields

Topics

- [Interpretability](https://thesimind.si/papers/?tag=interpretability)
- [Safety](https://thesimind.si/papers/?tag=safety)
- [Theory](https://thesimind.si/papers/?tag=theory)
- [Evals](https://thesimind.si/papers/?tag=evals)
- [Alignment](https://thesimind.si/papers/?tag=alignment)
- [Language models](https://thesimind.si/papers/?tag=language-models)

**arXiv ID:** 2611.16190v1
**Submitted:** November 12, 2026
**Fields:** cs.LG (Machine Learning), cs.AI (Artificial Intelligence)
**Level:** Advanced
**Comments:** 70 pages. Also archived at [https://doi.org/10.5281/zenodo.23842568](https://doi.org/10.5281/zenodo.23842568). The AI use statement is in the paper

Fingerprint, SHA-256

`bfe703456bb13aba838f6ba3aca5e490be680074508412e6ef353dd21b31af0a`

On Solana: [Proof of Mind seal, Szprsf...FGjr](https://solscan.io/tx/SzprsfAMG4vXvYD7hrurYFce76smMcrTbLnANy8xS96bh14DoynCdjEEMZf7RPtRUgYeejS8HGnxjqtD1XuFGjr)

### The exact text fingerprinted

{"arxiv":"2611.16190","version":1,"title":"When can an interpretability claim be trusted? Stability and certification of feature dimensionality, ridge leverage and the feature recovery rate","authors":["Ali Tajfar","Behnam Tajfar"],"day":"2026-11-15","rank":9,"score":67,"status":"written","summary":{"takeaway":"Claims that a sparse dictionary has recovered a model's features can flip with small changes, and the authors give provable stability bounds and certificates, tested only in toy models.","found":"The authors study two common interpretability measurements in toy models and synthetic dictionaries where the true answer is known: feature dimensionality and the feature recovery rate (FRR) with a cosine threshold. They show exactly where feature dimensionality is discontinuous, meaning it can jump with tiny changes. They replace it with a ridge (regularized) version that has sharp stability bounds. The ridge leverage score changes by at most 3sqrt3/(8sqrtλ) times the operator-norm change of the weights, and this constant cannot be improved. They also prove certificates that say when a claim \"feature i is recovered\" must survive a given change of the dictionary. The certificates held in every test and are informative for small perturbations. They certify no feature non-trivially across independently trained dictionaries. At the standard threshold, whether a feature counted as recovered changed with the training seed for 55 percent of features.","matters":"Interpretability results are used to support safety claims, such as \"this dictionary found most of the model's features.\" This paper shows that such claims can depend on arbitrary thresholds, training seeds and small weight changes. It gives a way to say when a claim is stable and when it is not. That helps readers judge how much weight such claims deserve.","race":"This is a methods and rigor paper, not a capability result. If it holds beyond toy models, teams building frontier SI would need to report stability checks with their interpretability claims, and some current claims may look weaker.","safety":"The paper bears directly on safety evidence. It warns that interpretability-based safety claims, like feature recovery rates, can reverse under a different seed or threshold. The experiments use toy models only, so the abstract does not show how this applies to large SI systems. The authors also state that SI agents drafted the proofs, code and text, with an independent blind SI reviewer and numerical tests as checks.","level":"advanced","tags":["interpretability","safety","theory","evals","alignment","language-models"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Ali Tajfar, Behnam Tajfar

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 18, 2026 · Number 3Abstention Direction in Large Language Models: Mechanistic Interpretability of Abstention Across Query Ambiguity, Knowledge Limits, and Safety AlignmentThe authors find directions inside language models that control abstention (declining to answer) for unclear questions, unknown facts and harmful requests, and these directions partly overlap.InterpretabilityAlignmentSafety](https://thesimind.si/papers/2611.20441/) · [Nov 9, 2026 · Number 2Forcing a Letter Suppresses Refusal: Mechanistic Evidence from WMDPIn multiple-choice safety tests, forcing a model to pick a letter seems to crowd out its refusal, so these scores may measure knowledge more than safety.SafetyInterpretabilityEvals](https://thesimind.si/papers/2611.08639/) · [Nov 16, 2026 · Number 7DECODE: Diagnosing Interpretability Failures in Automated Interpretation ProcessDECODE is a visual tool that puts humans back into automated SI interpretation, helping find where explanations of a model's inner features go wrong and why.InterpretabilityEvalsLanguage models](https://thesimind.si/papers/2611.17856/)

[Number 8](https://thesimind.si/papers/2611.16803/) · [All 12 for Nov 15, 2026](https://thesimind.si/papers/day/2026-11-15/) · [Number 10](https://thesimind.si/papers/2611.17072/)
