What they found
The authors built a dataset called Less is Safer (LIS) with only 2.4K samples. They picked prompts using Sparse Autoencoder (SAE) activations, which are readable signals inside a model, so the prompts cover many different safety features. They then used a multi-agent adversarial method to write harmless prompts that still trigger needless refusals. According to the abstract, LIS beats larger datasets, including TRIDENT-Core (26K), TRIDENT-Edge (19K) and WildJailbreak (262K), in overall performance on four safety benchmarks and five over-refusal benchmarks. Ablations showed that safety data alone or over-refusal data alone did not give the best results. A simpler filter that used LLM-annotated SAE safety features did worse than the authors' full pipeline.
Why it matters
Safety training usually needs large datasets, which costs money and often makes models refuse harmless requests. If a small, well-chosen set can do better, safety alignment gets cheaper and easier to repeat. The work also shows one way to use model interpretability tools to guide data choices. The abstract gives no details on which models were tested or how large the gains were, so the strength of the result is unclear.
How it changes the race
If the result holds, smaller teams could do safety alignment with far less data and compute. It also hints that the quality and coverage of safety data matter more than its size.
Safety and control
The paper targets two safety problems together: harmful outputs and over-refusal of harmless requests. It shows interpretability features can help guide alignment data. The abstract does not say how well this holds against new attacks.
The abstract, by the authors
Existing large language models typically rely on massive fine-tuning datasets for safety alignment, which results in higher costs for safety alignment and often leads to the problem of over-refusal. We propose Less is Safer (LIS), a compact yet highly effective dataset for safety alignment, constructed through interpretable feature-guided selection and adversarial synthesis. Specifically, we leverage Sparse Autoencoder (SAE) activations to efficiently select prompts that cover diverse and interpretable safety features. Then, we employ a multi-agent adversarial synthesis framework to generate harmless prompts that nevertheless elicit unnecessary refusals. LIS contains only 2.4K samples, yet outperforms much larger datasets, e.g., TRIDENT-Core (26K), TRIDENT-Edge (19K) and WildJailbreak (262K), in overall performance on four safety and five over-refusal benchmarks. Through ablation experiments, we demonstrate that neither safety data nor over-refusal data alone can achieve optimal results. We also used safety features from SAE, annotated by an LLM, for data filtering, but the results were inferior to those obtained using the pipeline proposed in this paper.
Quoted from arXiv:2611.16183. The research and its words belong to its authors.
