Hair-Trigger Alignment: Black-Box Evaluation Cannot Guarantee Post-Update Alignment

Large Language Models (LLMs) are rarely static and are frequently updated in practice. A growing body of alignment research has shown that models initially deemed ``aligned'' can exhibit misaligned behavior after fine-tuning. These works typically assume that the initial model is aligned based on static black-box evaluation, i.e., the absence of undesired responses to a fixed set of queries. However, the limits of black-box evaluation for post-update scenarios is not explored sufficiently. In this work, we formalize model alignment in both the static and post-update settings and uncover a fundamental limitation of black-box evaluation. We theoretically show that, due to overparameterization, static alignment provides no guarantee of post-update alignment for any update dataset. Moreover, we prove that static black-box probing cannot distinguish a model that is genuinely post-update robust from one that conceals an arbitrary amount of adversarial behavior which can be activated by even a single benign gradient update. We further validate these findings empirically in LLMs across three core alignment domains: privacy, jailbreak safety, and behavioral honesty. We demonstrate the existence of LLMs that pass all standard black-box alignment tests, yet become severely misaligned after a single benign update. Finally, we show that the capacity to hide such latent adversarial behavior increases with model scale, confirming our theoretical prediction that post-update misalignment grows with the number of parameters. Together, our results highlight the inadequacy of static evaluation protocols and emphasize the urgent need for post-update--robust alignment evaluation. Code can be found here.

Large language models (LLMs) are constantly updated after deployment to improve performance or adapt to new tasks. A common assumption is that if a model appears safe, honest, and privacy-preserving during testing, it will remain that way after future updates. However, it is unclear whether current evaluation methods can actually guarantee this. In this work, we show both theoretically and experimentally that standard “black-box” evaluations—which only test a model’s visible responses—cannot reliably determine whether a model will stay aligned after being updated. We prove that a model can appear completely aligned during testing while secretly containing harmful behaviors that become active after only a single benign training update. We call this phenomenon hair-trigger alignment. To demonstrate this, we construct language models that pass conventional evaluations for safety, honesty, and privacy, yet become misaligned after one harmless gradient update. For example, they can begin revealing previously hidden information, produce harmful responses, or behave dishonestly. We also show that larger models can hide more of this latent behavior. These findings suggest that current evaluation practices may provide a false sense of security. Ensuring that AI systems remain aligned after updates will likely require new evaluation methods that explicitly test robustness to future model modifications.