Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steering

Large language models (LLMs) can be controlled at inference time through prompts (in-context learning) and internal activations (activation steering). Different accounts have been proposed to explain these methods, yet their common goal of controlling model behavior raises the question of whether these seemingly disparate methodologies can be seen as specific instances of a broader framework. Motivated by this, we develop a unifying, predictive account of LLM control from a Bayesian perspective. Specifically, we posit that both context- and activation-based interventions impact model behavior by altering its belief in latent concepts: steering operates by changing concept priors, while in-context learning leads to an accumulation of evidence. This results in a closed-form Bayesian model that is highly predictive of LLM behavior across context- and activation-based interventions in a set of domains inspired by prior work on many-shot in-context learning. This model helps us explain prior empirical phenomena - e.g., sigmoidal learning curves as in-context evidence accumulates--while predicting novel ones--e.g., additivity of both interventions in log-belief space, which results in distinct phases such that sudden and dramatic behavioral shifts can be induced by slightly changing intervention controls. Taken together, this work offers a unified account of prompt-based and activation-based control of LLM behavior, and a methodology for empirically predicting the effects of these interventions.

Large language models can be guided in different ways: we can give them examples in the prompt (also known as "in-context learning"), or we can directly intervene on internal neural activations. These two approaches are usually studied separately, which makes it hard to predict how a model will change its behavior as a result of these interventions. We show that both methods can be understood through the same simple idea: changes in the model's beliefs. Examples in the prompt act like evidence that accumulates over time, while internal activations nudge the model’s prior tendency toward or away from a behavior. We provide empirical evidence for this perspective, and develop a mathematical model that accurately predicts how language models respond as we vary both the number of in-context learning examples and the strength of activation interventions.