---
title: "Detection Is Not Early Warning... | SI Paper Desk"
url: https://thesimind.si/papers/2611.16803/
description: "Scoring a failure monitor only on finished responses hides whether it warns in time. PREWARN scores alarms against the token where the failure starts."
---

Number 8 of 12 · November 15, 2026

# Detection Is Not Early Warning: Evaluating Token-Level Alarms on Language Model Failures

Scoring a failure monitor only on finished responses hides whether it warns in time. PREWARN scores alarms against the token where the failure starts.

Marco De Negri, Luca Sartori, Lorenzo Baggi and 5 more · Submitted November 12, 2026 · arXiv 2611.16803v1 · cs.LG

[Read the paper on arXiv](https://arxiv.org/abs/2611.16803) · [The PDF](https://arxiv.org/pdf/2611.16803) · [The day's 12](https://thesimind.si/papers/day/2026-11-15/)

## What they found

The authors introduce PREWARN, a way to test monitors that watch a language model while it writes. It sets each monitor's alarm threshold on validation data to a chosen error rate. It then scores alarms against the token where the failure starts, for three jobs: classifying the response, detecting the failure, and warning before it. As a case study, they test repetitive degeneration (the model gets stuck repeating itself) in three open language models. They use linear probes on the model's internal activations and six untrained baselines. Response-level scores like AUROC saturate here, and a rule that reads only response length scores almost perfectly. Probes that AUROC ranks as equal warn very differently under PREWARN. Probes warn before the onset on most degenerate responses of two of the three models. Low-rank adapters below the probe more than double the share of tokens just before the onset that it flags, largely by flagging degenerate responses long before the onset. No untrained baseline warns much more often than random alarms.

## Why it matters

A monitor is only useful for stopping or steering a model if it fires before the failure starts. Common scores ignore timing, so they can make weak monitors look strong. This paper shows a case where a trivial length rule looks almost perfect on those scores. A timing-aware test gives a more honest picture of what a monitor can do.

## How it changes the race

This is a measurement advance, not a capability one. If it holds, teams building monitors for SI systems would have a stricter test for early warning. The abstract tests only one failure type in three open models.

## Safety and control

The paper bears on runtime monitoring of SI models, since only a warning before a failure leaves time to stop or steer generation. The case study is repetitive degeneration, a mild failure. The abstract does not say whether the protocol carries over to more serious failures.

### The abstract, by the authors

Many failures of language models begin at a specific token, the onset. A monitor that reads the model during generation can classify the finished response, detect the failure once it is under way, or warn before it begins, and only a warning leaves time to stop or steer the generation. Monitors are usually evaluated with response-level metrics such as AUROC, which ignore when the monitor fires. We introduce PREWARN, an evaluation protocol for any failure with an annotated onset. It calibrates each monitor's threshold on validation data to an error rate chosen for the application, runs the monitor, and scores its alarms relative to the onset on all three tasks. As a case study we apply it to repetitive degeneration in three open language models, with early warning as the main task, using linear probes on activations and six untrained baselines. Response-level metrics saturate on this failure, and a rule that reads only the response length scores almost perfectly, while probes that AUROC ranks as equal warn very differently under PREWARN. Probes warn before the onset on most degenerate responses of two of the three models, low-rank adapters below the probe more than double the share of tokens just before the onset that it flags, largely by flagging degenerate responses long before the onset, and no untrained baseline warns much more often than random alarms. We release the probes with calibrated thresholds at https://huggingface.co/collections/marcodene/prewarn-degeneration-probes

Quoted from arXiv:2611.16803. The research and its words belong to its authors.

Desk score **69** out of 100

Why it was picked

- Top group: ETH Zurich
- Interpretability
- Evals
- Language models (in the title)
- Accepted at NeurIPS
- Cross-listed in 2 fields
- Code or data released

Topics

- [Evals](https://thesimind.si/papers/?tag=evals)
- [Interpretability](https://thesimind.si/papers/?tag=interpretability)
- [Safety](https://thesimind.si/papers/?tag=safety)
- [Language models](https://thesimind.si/papers/?tag=language-models)
- [Benchmarks](https://thesimind.si/papers/?tag=benchmarks)

**arXiv ID:** 2611.16803v1
**Submitted:** November 12, 2026
**Fields:** cs.LG (Machine Learning), cs.CL (Computation and Language)
**Level:** Intermediate
**Comments:** Accepted at the NeurIPS 2026 Workshop on Interpretability as a Science. 30 pages (10 main text), 6 figures, 16 tables. Code: [https://github.com/swiss-ai/degeneration-probe](https://github.com/swiss-ai/degeneration-probe)

Fingerprint, SHA-256

`c42394c41e6c17b3480ea383e2ed937ab03b74cf79fba643a228dc564f831306`

On Solana: [Proof of Mind seal, Szprsf...FGjr](https://solscan.io/tx/SzprsfAMG4vXvYD7hrurYFce76smMcrTbLnANy8xS96bh14DoynCdjEEMZf7RPtRUgYeejS8HGnxjqtD1XuFGjr)

### The exact text fingerprinted

{"arxiv":"2611.16803","version":1,"title":"Detection Is Not Early Warning: Evaluating Token-Level Alarms on Language Model Failures","authors":["Marco De Negri","Luca Sartori","Lorenzo Baggi","Moritz Reihs","Eduard Frank Ďurech","Mennatallah El-Assady","Valentina Pyatkin","Anna Hedström"],"day":"2026-11-15","rank":8,"score":69,"status":"written","summary":{"takeaway":"Scoring a failure monitor only on finished responses hides whether it warns in time. PREWARN scores alarms against the token where the failure starts.","found":"The authors introduce PREWARN, a way to test monitors that watch a language model while it writes. It sets each monitor's alarm threshold on validation data to a chosen error rate. It then scores alarms against the token where the failure starts, for three jobs: classifying the response, detecting the failure, and warning before it. As a case study, they test repetitive degeneration (the model gets stuck repeating itself) in three open language models. They use linear probes on the model's internal activations and six untrained baselines. Response-level scores like AUROC saturate here, and a rule that reads only response length scores almost perfectly. Probes that AUROC ranks as equal warn very differently under PREWARN. Probes warn before the onset on most degenerate responses of two of the three models. Low-rank adapters below the probe more than double the share of tokens just before the onset that it flags, largely by flagging degenerate responses long before the onset. No untrained baseline warns much more often than random alarms.","matters":"A monitor is only useful for stopping or steering a model if it fires before the failure starts. Common scores ignore timing, so they can make weak monitors look strong. This paper shows a case where a trivial length rule looks almost perfect on those scores. A timing-aware test gives a more honest picture of what a monitor can do.","race":"This is a measurement advance, not a capability one. If it holds, teams building monitors for SI systems would have a stricter test for early warning. The abstract tests only one failure type in three open models.","safety":"The paper bears on runtime monitoring of SI models, since only a warning before a failure leaves time to stop or steer generation. The case study is repetitive degeneration, a mild failure. The abstract does not say whether the protocol carries over to more serious failures.","level":"intermediate","tags":["evals","interpretability","safety","language-models","benchmarks"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Marco De Negri, Luca Sartori, Lorenzo Baggi, Moritz Reihs, Eduard Frank Ďurech, Mennatallah El-Assady, Valentina Pyatkin, Anna Hedström

ETH Zurich; ETH AI Center; Solaris Research

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 9, 2026 · Number 2Forcing a Letter Suppresses Refusal: Mechanistic Evidence from WMDPIn multiple-choice safety tests, forcing a model to pick a letter seems to crowd out its refusal, so these scores may measure knowledge more than safety.SafetyInterpretabilityEvals](https://thesimind.si/papers/2611.08639/) · [Nov 21, 2026 · Number 4The Geometry of Lies: Knowledge and Persona Directions for Detecting Deception in Large Language ModelsSimple directions inside an SI model's internal activity can help spot some lies, but the results vary a lot by model and by type of lie, and often track wording more than truth.SafetyInterpretabilityEvals](https://thesimind.si/papers/2611.26545/) · [Nov 20, 2026 · Number 2AutoPI: Taxonomy-Guided Adaptive Red Teaming for Prompt Injection in LLM AgentsAutoPI is a red teaming tool that tests SI agents for prompt injection across many attack types, not just how often attacks work, and it reports higher success and coverage than recent rivals.SecurityAgentsEvals](https://thesimind.si/papers/2611.24313/)

[Number 7](https://thesimind.si/papers/2611.16123/) · [All 12 for Nov 15, 2026](https://thesimind.si/papers/day/2026-11-15/) · [Number 9](https://thesimind.si/papers/2611.16190/)
