Equivalence of Context and Parameter Updates in Modern Transformer Blocks

Recent research has established that the impact of context in a vanilla transformer can be represented implicitly by forming a token-dependent, rank-1 patch to its MLP weights. This work extends that foundational theory to the diverse architectures of modern Large Language Models. We first demonstrate a precise, analytical solution for a Gemma-style transformer block, proving that the entire effect of a context can be perfectly mapped to rank-1 patches on its MLP weight matrices and a patch to the RMSNorm scale. We then generalize this result, providing a constructive proof and algorithm for multi-layer models. To unify these findings, we introduce a general framework centered on two core properties: input controllability and output controllability. We prove that a perfect implicit weight patch is possible for any MLP block where the inner function is input-controllable and the outer function is output-controllable. This provides a simpler and more powerful lens for understanding how transformer models transmute prompts into effective weights. This setup generalizes to a wide range of modern LLM architectures including gating, pre-/post-norm, mixture of experts and sequential/parallel transformer blocks.

Today's large language models can pick up new tasks just from reading a prompt, without any retraining. How does reading a few sentences change what a model does? One promising idea is that absorbing a prompt is equivalent to silently editing the model's internal weights, as if the prompt temporarily rewires the network for the task at hand. Earlier work proved this for a toy version of a transformer, but the systems actually used today have a more elaborate design, and it wasn't known whether the idea still held. We close that gap. We prove mathematically that for these modern designs, the effect of any prompt can be reproduced exactly by a small, precise edit to a handful of the model's internal weights, and we give an algorithm for computing it. Testing on Gemma 3 and Falcon, we find that a model given no prompt (but with our computed edits applied) produces, word for word, the same output as the original model reading the full prompt. This offers a clearer mathematical picture of what prompting really is which is a step toward more interpretable and trustworthy AI.