Detecting the Semantic Fixed Point: A Geometric Framework for Efficient Inference

Each layer of a Transformer refines the hidden state toward a prediction, an iterative process resembling fixed-point iteration. Yet when should this iteration terminate? Existing early exit methods rely on output confidence as a proxy for internal convergence. We take a more direct approach by examining the geometry of the hidden state trajectory. We find that layer-wise updates exhibit a two-phase structure: large, volatile updates in early layers, followed by small, aligned updates as the model propagates an already-formed representation. The transition is remarkably sharp. This yields a simple criterion: exit when step size vanishes and direction stabilizes. We track the normalized update norm and cosine similarity between consecutive updates, exiting when both indicate convergence. The overhead is $O(d)$ per layer, independent of vocabulary size, requiring no learned components or architectural modifications. On LLaMA-2-7B and LLaMA-2-13B across question answering and commonsense reasoning tasks, this geometric criterion reduces FLOPs by 30--35\% while retaining over 98\% of full-depth accuracy.

Large language models answer every question by passing it through all of their layers, even when the answer is already clear, and this wastes time and energy. We asked a simple question: can a model know when it has finished thinking? To find out, we looked at how the model's internal state changes from layer to layer, and we found a clear pattern. In the early layers the state changes a lot, but after a certain point it barely moves and keeps heading in the same direction, which means the model has settled on its answer and is just carrying it forward. So we let the model stop as soon as its state stops changing. This simple rule needs no extra training and almost no extra computation, yet it cuts the work by about a third while keeping nearly all of the accuracy, making large models cheaper and greener to run.