Number 8 of 12 · November 16, 2026

Safety Side Effects of Preventative Persona Steering: An Empirical Audit

Adding a persona vector during fine-tuning keeps MMLU accuracy, but the authors find it still shifts refusal behavior, so it should not be assumed safe without safety tests.

What they found

The authors audited preventative steering. This method adds a persona vector (a direction in the model's internal activity linked to a trait like sycophancy or hallucination) during fine-tuning, then removes it at evaluation. They tested sycophancy and hallucination vectors on three models. They measured three safety benchmarks, plus trait expression, coherence and MMLU. They report that preventative steering still changes refusal behavior, by an amount that depends on the trait. The abstract does not give the sizes of the changes or the direction.

Why it matters

Earlier work by Chen et al. (2025) recommended preventative steering because it keeps MMLU accuracy. The authors say earlier checks of that advice looked at capability only. This paper adds safety measurements. It suggests that keeping accuracy is not enough to call a method safe.

How it changes the race

This is a small, focused check, not a new method. It asks teams that use persona steering to run several safety benchmarks next to capability tests before adopting it.

Safety and control

The result bears on safety. A technique meant to reduce bad traits can shift refusal behavior as a side effect. The authors say it does not establish that preventative steering is safety-neutral, and that steering methods should be tested on several safety benchmarks.

The abstract, by the authors

High-level behaviors such as sycophancy and hallucination correspond to directions in a model's activation space (Chen et al., 2025). These directions, known as persona vectors, can suppress undesirable traits either at inference time, subtracting the vector during generation, or through preventative steering, adding the vector during fine-tuning, so the optimizer never needs to encode the trait in weights, then removing the vector at evaluation. Chen et al. (2025) recommend preventative steering because it preserves Massive Multitask Language Understanding (MMLU) accuracy, but validations of this recommendation have measured capability alone. We primarily evaluate results on preventative steering of sycophancy and hallucination vectors on three models, measuring three safety benchmarks alongside trait expression, coherence, and MMLU. Our findings show that preventative steering still changes refusal behavior, but by an amount that depends on the trait and does not establish that preventative persona steering is safety-neutral, and steering methods should be evaluated on several safety benchmarks.

Quoted from arXiv:2611.17620. The research and its words belong to its authors.

The authors

Sabrina Tang, Nikolai Vo, Carson Murillo, Chira Sukumarnpiti, Sarah Wilson

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.