SI Safety Moves From Scores to Process and Evidence
This week's papers keep returning to one question: how do we know what an SI system is really doing? Phantom Consensus shows multi-agent models can drift to a group answer no one privately held, and checking private judgments can catch it. Forcing a Letter Suppresses Refusal suggests multiple-choice safety scores may measure knowledge more than safety. Contain, Conform, Compare adds that several apparent gains did not hold up on new tasks. In the same vein, the comparative study of agent evaluation frameworks found that the method seeing the full step-by-step trace caught failures best, and MobileProbe reports that adapting attacks to each agent's route beats fixed tests. On the training side, META-IR, the data-efficient alignment work using 2.4K examples, and the "Going Badly" activations result all look inside the model, though the last one differs across models. Worth watching next: whether trace-level monitors, as in the evaluation-side game paper, and defenses such as Bait and TAS hold up beyond small models and controlled tests.
