Steering Beyond the Support: Adversarial Training on Unsupervised Jailbroken Activation Simulation

Jailbreak prompts can trigger harmful completions on aligned LLMs, In accordance, safety steering has been proposed: test-time activation interventions that steer jailbreak activations to trigger refusal while preserving benign utility. However, existing steering methods are fundamentally supervised and tied to a static, limited training set, whereas real jailbreaks evolve and are often out-of-distributed from the training set, leading to failures on unseen attacks. In this paper, we tackle this failure by developping a \emph{zero-shot} defense. Base on unsupervised latent direction discovery, we directly simulate jailbroken activations without any knowledge of jailbreak strategy. To build a defense mechnism upon this, we propose a bi-level adversarial training framework. In the inner step, we simulate diverse jailbroken activations by extrapolating from refusal state harmful-request activations via unsupervised latent direction discovery. In the outer step, we train a potential-induced steering field to push these adversarial jailbroken states into refusal regions while keeping benign unchanged. Across three LLMs and six classical jailbreak families, our method achieves strong defense with attack success rates mostly below 5%, and we analyzed the increasing subspace coverage of our simulated jailbroken activations on real jailbreaks throughout training, which helps explain the increasing robustness of our defense mechnism.

People can trick otherwise safety-trained language models into giving harmful answers by wrapping dangerous requests in clever prompts, a practice often called jailbreak prompting. Existing defenses usually learn from a fixed set of known attacks, but new attacks can look very different from those examples and may bypass the defense. We propose a way to prepare the defense for attacks it has never seen by creating simulated ``unsafe'' internal states inside the model, without needing examples of specific jailbreak prompts. Our method repeatedly generates these challenging internal states and trains a lightweight steering mechanism to move them back toward safe refusal behavior, while leaving normal helpful requests largely unchanged. Across three widely used language models and six types of jailbreak attacks, this approach greatly reduces harmful responses, often bringing attack success rates below 5\%. This research matters because it suggests a path toward language-model safety tools that are less tied to yesterday's attacks and more robust to the new tricks people may invent tomorrow.