Latent Reasoning in TRMs is Secretly a Policy Improvement Operator

Recently, small models with latent recursion have obtained promising results on complex reasoning tasks. These results are typically explained by the theory that such recursion increases a network’s depth, allowing it to compactly emulate the capacity of larger models. However, the performance of recursively added layers remains behind the capabilities of one‑pass models with the same feed-forward depth. This means that in the looped version, not every recursive step effectively contributes to depth. This raises the question: when and why does latent reasoning improve performance, and when does it result in dead compute? In our work, we analyze the algorithms that latent reasoning provides answer to this question. We show that latent reasoning can be formalized as a classifier‑free guidance and policy improvement algorithm. Building on these insights, we propose to use a training schemes from RL and diffusion methods for latent reasoning modles. Using the Tiny Recursive Model as our testbed, we show that with our modifications we can avoid dead compute steps and reduce the total number of forward passes by 18× while maintaining performance. Broadly speaking, we show how a policy improvement perspective on recursive steps can explain model behavior and provide insights for further improvements.

Small reasoning models can sometimes solve difficult puzzles by applying the same network repeatedly, refining their answer step by step. However, it is often unclear why these repeated steps help. In many cases, extra computation may be wasted because some steps do not meaningfully improve the answer. We show that the repeated reasoning steps used by Tiny Recursive Models can be understood as a form of policy improvement: each step starts from the model’s current guess and should move it toward a better one. Based on this view, we introduce Deep Improvement Supervision, a training method that gives the model useful intermediate targets instead of only supervising the final answer. These targets are created by gradually reducing corruption in the correct output, so every recursion step learns a clear sub-goal. This makes small recursive reasoning models more efficient and easier to train. On puzzle-like reasoning benchmarks such as N-Queens and ARC-AGI, our method maintains strong performance while using far fewer reasoning steps and forward passes than the original Tiny Recursive Model. This suggests a path toward compact AI systems that reason more deliberately, with less wasted computation.