Wait, Wait, Wait... Why Do Reasoning Models Loop?

Reasoning models (e.g., DeepSeek-R1) generate long chains of thought to solve harder problems, but they often loop, repeating the same text at low temperatures or with greedy decoding. We study why this happens and what role temperature plays. With open reasoning models, we find that looping is common at low temperature. Larger models tend to loop less, and distilled students loop significantly even when their teachers rarely do. This points to mismatches between the training distribution and the learned model, which we refer to as errors in learning, as a key cause. To understand how such errors cause loops, we introduce a synthetic graph reasoning task and demonstrate two mechanisms. First, risk aversion caused by hardness of learning: when the correct progress-making action is hard to learn but an easy cyclic action is available, the model puts relatively more probability on the cyclic action and gets stuck. Second, even when there is no hardness, Transformers show an inductive bias toward temporally correlated errors, so the same few actions keep being chosen and loops appear. Higher temperature reduces looping by promoting exploration, but it does not fix the errors in learning, so generations remain much longer than necessary at high temperature; in this sense, temperature is a stopgap rather than a holistic solution. We end with a discussion of training-time interventions aimed at directly reducing errors in learning.

Modern reasoning language models sometimes get stuck in infinite loops — endlessly repeating the same phrase. Studying a range of such models, we find that smaller models loop more than larger ones, harder problems cause more looping, and smaller models trained on a larger model's outputs can loop far more than their teacher. This points to imperfect learning as a root cause. To understand how imperfect learning leads to looping, we build a synthetic graph reasoning task and identify two mechanisms. First, when a problem is hard, models exhibit risk aversion: rather than attempting a difficult next step, they default to an easy repetitive action like restating something they already said. Second, when models are faced with a decision point where multiple actions are equally valid, they tend to repeatedly bias toward the same action. The standard mitigation is to add randomness to the model's choices by increasing a temperature parameter, making it less deterministic. We show this reduces looping on the surface but does not fix the underlying causes. Addressing looping properly likely requires changes to how these models are trained, for instance by making training examples easier for smaller models to learn from.