A model of errors in transformers

We study the error rate of LLMs on tasks like arithmetic that require a deterministic output, and repetitive processing of tokens drawn from a small set of alternatives. We argue that incorrect predictions arise when small errors in the attention mechanism accumulate to cross a threshold, and use this insight to derive a quantitative two-parameter relationship between the accuracy and the complexity of the task. The two parameters vary with the prompt and the model; they can be interpreted in terms of an elementary noise rate, and the number of plausible erroneous tokens that can be predicted. Our analysis is inspired by an "effective field theory'' perspective: the LLM's many raw parameters can be reorganized into just two parameters that govern the error rate. We perform extensive empirical tests, using Gemini 2.5 Flash, Gemini 2.5 Pro and DeepSeek R1, and find excellent agreement between the predicted and observed accuracy for a variety of tasks, although we also identify deviations in some cases. Our model provides an alternative to suggestions that errors made by LLMs on long repetitive tasks indicate the "collapse of reasoning'', or an inability to express "compositional'' functions. Finally, we show how to construct prompts to reduce the error rate.

State-of-the-art large language models (LLMs) make errors on simple tasks such as arithmetic involving large numbers. Our objective in this paper was to develop a theoretical and quantitative model of these errors, as is done in physics. Our main insight is that even though an LLM is an enormously complicated system, while it is engaged in one of these tasks, its behavior can be understood using a much simpler computational process. We can then analyze how errors accumulate in this simpler process. This perspective led us to a precise formula that predicts how the accuracy of LLMs drops as the tasks involved become harder. We verified this formula by performing about two hundred thousand experiments across three state-of-the-art LLMs and eight different tasks. Our experiments bear out the theory nicely, and the one significant exception that we encountered suggested something interesting about the workings of one of the LLMs. We were also able to use our insights to design prompts that significantly improve accuracy. From an engineering perspective, AI systems have made enormous strides. Even the problems that we consider, such as arithmetic, can be solved in a practical setting by teaching LLMs to use auxiliary tools. But our theoretical understanding of these systems is in an incipient stage, when compared to mature natural sciences such as physics. We are hoping that our paper's successful analysis of the operation of LLMs in a simple setting will contribute to this broader theoretical program.