---
title: "Safety Side Effects of Preventative... | SI Paper Desk"
url: https://thesimind.si/papers/2611.17620/
description: "Adding a persona vector during fine-tuning keeps MMLU accuracy, but the authors find it still shifts refusal behavior, so it should not be assumed safe..."
---

Number 8 of 12 · November 16, 2026

# Safety Side Effects of Preventative Persona Steering: An Empirical Audit

Adding a persona vector during fine-tuning keeps MMLU accuracy, but the authors find it still shifts refusal behavior, so it should not be assumed safe without safety tests.

Sabrina Tang, Nikolai Vo, Carson Murillo and 2 more · Submitted November 13, 2026 · arXiv 2611.17620v1 · cs.AI

[Read the paper on arXiv](https://arxiv.org/abs/2611.17620) · [The PDF](https://arxiv.org/pdf/2611.17620) · [The day's 12](https://thesimind.si/papers/day/2026-11-16/)

## What they found

The authors audited preventative steering. This method adds a persona vector (a direction in the model's internal activity linked to a trait like sycophancy or hallucination) during fine-tuning, then removes it at evaluation. They tested sycophancy and hallucination vectors on three models. They measured three safety benchmarks, plus trait expression, coherence and MMLU. They report that preventative steering still changes refusal behavior, by an amount that depends on the trait. The abstract does not give the sizes of the changes or the direction.

## Why it matters

Earlier work by Chen et al. (2025) recommended preventative steering because it keeps MMLU accuracy. The authors say earlier checks of that advice looked at capability only. This paper adds safety measurements. It suggests that keeping accuracy is not enough to call a method safe.

## How it changes the race

This is a small, focused check, not a new method. It asks teams that use persona steering to run several safety benchmarks next to capability tests before adopting it.

## Safety and control

The result bears on safety. A technique meant to reduce bad traits can shift refusal behavior as a side effect. The authors say it does not establish that preventative steering is safety-neutral, and that steering methods should be tested on several safety benchmarks.

### The abstract, by the authors

High-level behaviors such as sycophancy and hallucination correspond to directions in a model's activation space (Chen et al., 2025). These directions, known as persona vectors, can suppress undesirable traits either at inference time, subtracting the vector during generation, or through preventative steering, adding the vector during fine-tuning, so the optimizer never needs to encode the trait in weights, then removing the vector at evaluation. Chen et al. (2025) recommend preventative steering because it preserves Massive Multitask Language Understanding (MMLU) accuracy, but validations of this recommendation have measured capability alone. We primarily evaluate results on preventative steering of sycophancy and hallucination vectors on three models, measuring three safety benchmarks alongside trait expression, coherence, and MMLU. Our findings show that preventative steering still changes refusal behavior, but by an amount that depends on the trait and does not establish that preventative persona steering is safety-neutral, and steering methods should be evaluated on several safety benchmarks.

Quoted from arXiv:2611.17620. The research and its words belong to its authors.

Desk score **71** out of 100

Why it was picked

- Safety (in the title)
- Alignment
- Evals
- Accepted at NeurIPS

Topics

- [Safety](https://thesimind.si/papers/?tag=safety)
- [Alignment](https://thesimind.si/papers/?tag=alignment)
- [Evals](https://thesimind.si/papers/?tag=evals)
- [Interpretability](https://thesimind.si/papers/?tag=interpretability)
- [Language models](https://thesimind.si/papers/?tag=language-models)
- [Training](https://thesimind.si/papers/?tag=training)

**arXiv ID:** 2611.17620v1
**Submitted:** November 13, 2026
**Fields:** cs.AI (Artificial Intelligence)
**Level:** Intermediate
**Comments:** 5 pages, 1 figure (6 charts), 1 table, accepted at NeurIPS 2026 Bialign Workshop

Fingerprint, SHA-256

`acf647eb6c3a1ab2f835400f792fe79d01da41bebbdcba7e2bec1cda8d72dcb5`

On Solana: [Proof of Mind seal, 3mXUMW...GSim](https://solscan.io/tx/3mXUMW5MkdFiyPBNPYd5DfwHhma5EdjhUFEw9jZaDuuPqgszJtoDYttSrD9kyWUFdMCzn7RnFNd4nQhwAUgrGSim)

### The exact text fingerprinted

{"arxiv":"2611.17620","version":1,"title":"Safety Side Effects of Preventative Persona Steering: An Empirical Audit","authors":["Sabrina Tang","Nikolai Vo","Carson Murillo","Chira Sukumarnpiti","Sarah Wilson"],"day":"2026-11-16","rank":8,"score":71,"status":"written","summary":{"takeaway":"Adding a persona vector during fine-tuning keeps MMLU accuracy, but the authors find it still shifts refusal behavior, so it should not be assumed safe without safety tests.","found":"The authors audited preventative steering. This method adds a persona vector (a direction in the model's internal activity linked to a trait like sycophancy or hallucination) during fine-tuning, then removes it at evaluation. They tested sycophancy and hallucination vectors on three models. They measured three safety benchmarks, plus trait expression, coherence and MMLU. They report that preventative steering still changes refusal behavior, by an amount that depends on the trait. The abstract does not give the sizes of the changes or the direction.","matters":"Earlier work by Chen et al. (2025) recommended preventative steering because it keeps MMLU accuracy. The authors say earlier checks of that advice looked at capability only. This paper adds safety measurements. It suggests that keeping accuracy is not enough to call a method safe.","race":"This is a small, focused check, not a new method. It asks teams that use persona steering to run several safety benchmarks next to capability tests before adopting it.","safety":"The result bears on safety. A technique meant to reduce bad traits can shift refusal behavior as a side effect. The authors say it does not establish that preventative steering is safety-neutral, and that steering methods should be tested on several safety benchmarks.","level":"intermediate","tags":["safety","alignment","evals","interpretability","language-models","training"]}}

[Check it on Verify a thought](https://thesimind.si/proof-of-mind/)

## The authors

Sabrina Tang, Nikolai Vo, Carson Murillo, Chira Sukumarnpiti, Sarah Wilson

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.

Related on the desk

## More papers on these topics

[Nov 18, 2026 · Number 3Abstention Direction in Large Language Models: Mechanistic Interpretability of Abstention Across Query Ambiguity, Knowledge Limits, and Safety AlignmentThe authors find directions inside language models that control abstention (declining to answer) for unclear questions, unknown facts and harmful requests, and these directions partly overlap.InterpretabilityAlignmentSafety](https://thesimind.si/papers/2611.20441/) · [Nov 9, 2026 · Number 2Forcing a Letter Suppresses Refusal: Mechanistic Evidence from WMDPIn multiple-choice safety tests, forcing a model to pick a letter seems to crowd out its refusal, so these scores may measure knowledge more than safety.SafetyInterpretabilityEvals](https://thesimind.si/papers/2611.08639/) · [Nov 16, 2026 · Number 6Alignment with Reflect-Reasoning: Learning How To Follow PrinciplesReflect-Reasoning trains models to reason about whether they are following a given set of principles, and the authors report it beats system prompting across five models.AlignmentReasoningLanguage models](https://thesimind.si/papers/2611.17398/)

[Number 7](https://thesimind.si/papers/2611.17856/) · [All 12 for Nov 16, 2026](https://thesimind.si/papers/day/2026-11-16/) · [Number 9](https://thesimind.si/papers/2611.17395/)
