Endogenous Resistance to Activation Steering in Language Models

Large language models can resist task-misaligned activation steering during inference, sometimes recovering mid-generation to produce improved responses even when steering remains active. We term this Endogenous Steering Resistance (ESR). Using sparse autoencoder (SAE) latents to steer model activations, we find that Llama-3.3-70B shows substantial ESR, while smaller models from the Llama-3 and Gemma-2 families exhibit the phenomenon less frequently. We identify 26 ``off-topic detector'' latents that predict ESR episodes in Llama-3.3-70B. Zero-ablating these latents reduces the multi-attempt rate by 25\%, providing causal evidence for dedicated internal consistency-checking circuits. We demonstrate that ESR can be deliberately enhanced through both prompting and training: meta-prompts instructing the model to self-monitor increase the multi-attempt rate by 5$\times$ for Llama-3.3-70B, and fine-tuning on self-correction examples successfully induces ESR-like behavior in smaller models. These findings have dual implications: ESR could protect against adversarial manipulation but might also interfere with beneficial safety interventions that rely on activation steering. Understanding and controlling these resistance mechanisms is important for developing transparent and controllable AI systems.

When a language model answers a question, it relies on patterns of activity inside the model. Researchers can deliberately interfere with these patterns to nudge the model toward a chosen topic, a technique called activation steering. We study what happens when this nudge conflicts with the user’s actual request. Surprisingly, the largest model we tested, Llama-3.3-70B, sometimes notices the problem mid-answer, writes something like “wait, that’s not right,” and returns to the original task even though the steering remains active. We call this behavior Endogenous Steering Resistance. Smaller Llama and Gemma models rarely show this explicit form of recovery. Our experiments suggest that the visible restart is often triggered by the model reading its own off-topic text, but that staying on-topic afterward cannot be fully explained by ordinary text continuation alone. We also identify model-internal signals associated with this recovery: removing them makes successful restarts less common. These findings matter for AI safety because steering resistance could help models withstand malicious internal manipulation, but could also make beneficial safety interventions harder to apply.