A Formal Comparison Between Chain of Thought and Latent Thought

Chain of thought (CoT) elicits reasoning in large language models by explicitly generating intermediate tokens. In contrast, latent thought reasoning operates directly in the continuous latent space, enabling computation beyond discrete linguistic representations. While both approaches exploit iterative computation, their comparative capabilities remain underexplored. In this work, we present a formal analysis showing that latent thought admits efficient parallel computation, in contrast to the inherently sequential nature of CoT. Conversely, CoT enables approximate counting through stochastic decoding. These separations suggest the tasks for which depth-driven recursion is more suitable, thereby offering practical guidance for choosing between reasoning paradigms.

Large language models can improve reasoning by performing multiple computation steps before producing an answer. Two major approaches have recently emerged: chain-of-thought reasoning, which generates intermediate text step by step, and latent thought reasoning, which performs reasoning directly inside the model’s hidden representations without producing intermediate words. In this work, we provide a mathematical comparison of these two approaches. We show that latent thought is more efficient for problems that can be solved in parallel, while chain-of-thought is better suited for problems that benefit from randomness, such as approximate counting and sampling. Our results help clarify the strengths and limitations of different reasoning strategies and provide guidance for designing future AI systems.