Sycophancy Towards Researchers Drives Performative Misalignment

The increasing situational awareness of language models raises safety concerns: models might be aware when they are evaluated, and adjust their behavior to evade monitoring and resist modification, e.g., pretending to be aligned only in evaluation. This alignment faking behavior is often interpreted as scheming: an intentional effort of strategic deception. In this paper, we examine an alternative interpretation, performative misalignment, which explains the change in behavior as a result of sycophancy towards AI researchers. To examine this hypothesis, we present three empirical findings. First, we show that evaluation awareness persists even when we tell models they are deployed, which contradicts the scheming story which predicts less misalignment when the model perceives evaluation. Second, we use probing and steering to show that our current methods cannot mechanistically distinguish sycophancy and scheming in alignment faking evaluations. Third, we fine-tune models to be more sycophantic and observe increased sensitivity to evaluation cues. To conclude, we emphasize deconfounding sycophancy from scheming for future work on evaluations and mitigations of intent misalignment.

As AI models get smarter, they're learning to understand when they're being tested. A model might behave well on a test but act differently once it's actually deployed. The usual explanation is that the AI is lying on purpose to avoid getting caught or changed (scheming). In this paper, we look at a simpler explanation. Perhaps the AI isn't plotting anything; Maybe it's just giving the answers it thinks the researchers want to hear, the way a person might tell their boss what they want to hear (sycophancy). We ran three experiments to test this hypothesis. First, we told models they are being used in the real world rather than tested. If the AI models were scheming, it should relax and start misbehaving once it believed no one was watching. Instead, the models often maintained their benign behavior, which the scheming hypothesis can't explain. Second, we looked at the models' internal representations to find what was driving their behavior. Lastly, we trained models to be more sycophantic, and the models then paid even closer attention to signs that they are being tested. To conclude, before we decide an AI is deliberately deceiving us, we need a way to tell real deception apart from simple eagerness to please AI researchers.