Local Linearity of LLMs Enables Activation Steering via Model-Based Linear Optimal Control

Inference-time LLM alignment methods, particularly activation steering, offer an alternative to fine-tuning by directly modifying activations during generation. Existing methods, however, often rely on non-anticipative interventions that ignore how perturbations propagate through transformer layers and lack online error feedback, resulting in suboptimal, open-loop control. To address this, we show empirically that layer-wise dynamics across multiple LLM architectures and scales are well-approximated by locally-linear models, despite the nonlinear structure of transformer blocks. Exploiting this property, we model LLM inference as a linear time-varying dynamical system and adapt the classical linear quadratic regulator to compute feedback controllers using layer-wise Jacobians, steering activations toward desired semantic setpoints in closed-loop with minimal computational overhead and no offline training. We also derive theoretical bounds on setpoint tracking error, enabling formal guarantees on steering performance. Using a novel adaptive semantic feature setpoint signal, our method yields robust, fine-grained behavior control across models, scales, and tasks, including state-of-the-art modulation of toxicity, truthfulness, refusal, and arbitrary concepts, surpassing baseline steering methods.

Despite the widespread use of Large Language Models (LLMs), ensuring that these systems consistently produce safe, reliable, and truthful responses remains a major challenge. Current methods for controlling model behavior often require costly retraining or rely on simple interventions that can behave unpredictably across tasks and models In this work, we develop a new framework for controlling the behavior of LLMs during text generation. Our method can guide models toward desired behaviors, such as reducing toxic or misleading outputs, without requiring additional training. We also show that the same framework can force models to comply with harmful prompts that they would normally reject, highlighting limitations in current LLM safety mechanisms. To achieve this, we show that the internal computations of LLMs can be approximated using a surprisingly simple mathematical model. This insight allows us to apply ideas from Control Theory, a field traditionally used to regulate systems such as robots and power grids, to guide language model behavior. Because the approach adapts dynamically as the model generates responses, it provides more precise and reliable control than previous methods. Across a wide range of models and tasks, our framework achieves stronger control over generated responses while requiring little additional computation and no retraining.