---
title: "Data-Efficient LLM Safety Alignment... | SI Paper Desk"
url: https://thesimind.si/papers/2611.16183/
description: "A small, carefully chosen set of 2.4K training examples, picked using interpretable features inside a model, beat much larger safety datasets on safety and..."
---

Paper of the Day · November 14, 2026

# Data-Efficient LLM Safety Alignment Guided by Interpretable Features

A small, carefully chosen set of 2.4K training examples, picked using interpretable features inside a model, beat much larger safety datasets on safety and over-refusal tests.

Hongyu Li, Zhan Yu, Zhe Zhao and 3 more · Submitted November 12, 2026 · arXiv 2611.16183v1 · cs.CL

[Read the paper on arXiv](https://arxiv.org/abs/2611.16183) · [The PDF](https://arxiv.org/pdf/2611.16183) · [The day's 12](https://thesimind.si/papers/day/2026-11-14/)

## What they found

The authors built a dataset called Less is Safer (LIS) with only 2.4K samples. They picked prompts using Sparse Autoencoder (SAE) activations, which are readable signals inside a model, so the prompts cover many different safety features. They then used a multi-agent adversarial method to write harmless prompts that still trigger needless refusals. According to the abstract, LIS beats larger datasets, including TRIDENT-Core (26K), TRIDENT-Edge (19K) and WildJailbreak (262K), in overall performance on four safety benchmarks and five over-refusal benchmarks. Ablations showed that safety data alone or over-refusal data alone did not give the best results. A simpler filter that used LLM-annotated SAE safety features did worse than the authors' full pipeline.

## Why it matters

Safety training usually needs large datasets, which costs money and often makes models refuse harmless requests. If a small, well-chosen set can do better, safety alignment gets cheaper and easier to repeat. The work also shows one way to use model interpretability tools to guide data choices. The abstract gives no details on which models were tested or how large the gains were, so the strength of the result is unclear.

## How it changes the race

If the result holds, smaller teams could do safety alignment with far less data and compute. It also hints that the quality and coverage of safety data matter more than its size.

## Safety and control

The paper targets two safety problems together: harmful outputs and over-refusal of harmless requests. It shows interpretability features can help guide alignment data. The abstract does not say how well this holds against new attacks.

### The abstract, by the authors

Existing large language models typically rely on massive fine-tuning datasets for safety alignment, which results in higher costs for safety alignment and often leads to the problem of over-refusal. We propose Less is Safer (LIS), a compact yet highly effective dataset for safety alignment, constructed through interpretable feature-guided selection and adversarial synthesis. Specifically, we leverage Sparse Autoencoder (SAE) activations to efficiently select prompts that cover diverse and interpretable safety features. Then, we employ a multi-agent adversarial synthesis framework to generate harmless prompts that nevertheless elicit unnecessary refusals. LIS contains only 2.4K samples, yet outperforms much larger datasets, e.g., TRIDENT-Core (26K), TRIDENT-Edge (19K) and WildJailbreak (262K), in overall performance on four safety and five over-refusal benchmarks. Through ablation experiments, we demonstrate that neither safety data nor over-refusal data alone can achieve optimal results. We also used safety features from SAE, annotated by an LLM, for data filtering, but the results were inferior to those obtained using the pipeline proposed in this paper.

Quoted from arXiv:2611.16183. The research and its words belong to its authors.

Desk score **92** out of 100

Why it was picked

- Safety (in the title)
- Alignment (in the title)
- Interpretability (in the title)
- Agents
- Evals
- Language models (in the title)
- Multi-agent

Topics

- [Alignment](https://thesimind.si/papers/?tag=alignment)
- [Interpretability](https://thesimind.si/papers/?tag=interpretability)
- [Safety](https://thesimind.si/papers/?tag=safety)
- [Data](https://thesimind.si/papers/?tag=data)
- [Efficiency](https://thesimind.si/papers/?tag=efficiency)
- [Language models](https://thesimind.si/papers/?tag=language-models)

**arXiv ID:** 2611.16183v1
**Submitted:** November 12, 2026
**Fields:** cs.CL (Computation and Language), cs.AI (Artificial Intelligence)
**Level:** Intermediate
**Comments:** 36 pages, 16 figures

Fingerprint, SHA-256

`35b82c052a4e8ad2306b87441d0aa3ad3ec166f5de6a5673c11b9b1bb4c2f24a`

On Solana: [Proof of Mind seal, 4ut3vv...Y9uP](https://solscan.io/tx/4ut3vv2vTeUs5fAbknPuGnt1i1jJGh18Ry7QYkUAkvvDWCKQk8wrmGNkp4C5Loc937KnsizybKH71BB2w6QAY9uP)

### The exact text fingerprinted

{"arxiv":"2611.16183","version":1,"title":"Data-Efficient LLM Safety Alignment Guided by Interpretable Features","authors":["Hongyu Li","Zhan Yu","Zhe Zhao","Kimmo Yan","Xiong Kuang","Renfen Hu"],"day":"2026-11-14","rank":1,"score":92,"status":"written","summary":{"takeaway":"A small, carefully chosen set of 2.4K training examples, picked using interpretable features inside a model, beat much larger safety datasets on safety and over-refusal tests.","found":"The authors built a dataset called Less is Safer (LIS) with only 2.4K samples. They picked prompts using Sparse Autoencoder (SAE) activations, which are readable signals inside a model, so the prompts cover many different safety features. They then used a multi-agent adversarial method to write harmless prompts that still trigger needless refusals. According to the abstract, LIS beats larger datasets, including TRIDENT-Core (26K), TRIDENT-Edge (19K) and WildJailbreak (262K), in overall performance on four safety benchmarks and five over-refusal benchmarks. Ablations showed that safety data alone or over-refusal data alone did not give the best results. A simpler filter that used LLM-annotated SAE safety features did worse than the authors' full pipeline.","matters":"Safety training usually needs large datasets, which costs money and often makes models refuse harmless requests. If a small, well-chosen set can do better, safety alignment gets cheaper and easier to repeat. The work also shows one way to use model interpretability tools to guide data choices. The abstract gives no details on which models were tested or how large the gains were, so the strength of the result is unclear.","race":"If the result holds, smaller teams could do safety alignment with far less data and compute. It also hints that the quality and coverage of safety data matter more than its size.","safety":"The paper targets two safety problems together: harmful outputs and over-refusal of harmless requests. It shows interpretability features can help guide alignment data. The abstract does not say how well this holds against new attacks.","level":"intermediate","tags":["alignment","interpretability","safety","data","efficiency","language-models"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Hongyu Li, Zhan Yu, Zhe Zhao, Kimmo Yan, Xiong Kuang, Renfen Hu

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 18, 2026 · Number 3Abstention Direction in Large Language Models: Mechanistic Interpretability of Abstention Across Query Ambiguity, Knowledge Limits, and Safety AlignmentThe authors find directions inside language models that control abstention (declining to answer) for unclear questions, unknown facts and harmful requests, and these directions partly overlap.InterpretabilityAlignmentSafety](https://thesimind.si/papers/2611.20441/) · [Nov 19, 2026 · Number 2Runtime Verification of Multiple Natural Language Criteria for Agent GovernanceThe authors built VFM, a small verifier that checks an SI agent's output against many written rules at once and says which are met, broken, or not relevant.SafetyAgentsEvals](https://thesimind.si/papers/2611.23074/) · [Nov 9, 2026 · Number 2Forcing a Letter Suppresses Refusal: Mechanistic Evidence from WMDPIn multiple-choice safety tests, forcing a model to pick a letter seems to crowd out its refusal, so these scores may measure knowledge more than safety.SafetyInterpretabilityEvals](https://thesimind.si/papers/2611.08639/)

[All 12 for Nov 14, 2026](https://thesimind.si/papers/day/2026-11-14/) · [Number 2](https://thesimind.si/papers/2611.16369/)
