How Far Ahead Do LLMs Plan? Uncovering the Latent Horizon in Chain-of-Thought Reasoning

Chain-of-thought (CoT) reasoning has become a central mechanism for eliciting multi-step reasoning in Large Language Models (LLMs). Yet recent evidence presents a tension: hidden states appear to already encode future reasoning before CoT fully unfolds, while explicit steps still remain crucial for tasks requiring compositional computation. To deepen the understanding between LLM's internal states and its verbalized reasoning trajectories, we investigate the latent planning strength of LLMs, through our probing method, Tele-Lens, applying to hidden states across diverse task domains. Our empirical results indicate that LLMs exhibit a myopic horizon, primarily conducting incremental transitions without precise global planning. Leveraging this characteristic, we propose a hypothesis on enhancing uncertainty estimation of CoT, which we validate that a sparse set of pivot positions can effectively represent the uncertainty of the entire path. We further underscore the significance of exploiting CoT dynamics, and demonstrate that automatic recognition of CoT bypass can be achieved without performance degradation. Our code, data and models are released at https://github.com/lxucs/tele-lens.

Large language models often appear to “think” by writing step-by-step explanations before giving an answer, but it is unclear whether they already know the full plan in advance or are reasoning as they go. This paper studies how far ahead these models can internally plan during chain-of-thought reasoning. We introduce a probing method called Tele-Lens, which looks inside a model’s hidden states during reasoning and tests whether they contain information about future steps, final answers, and reasoning length. Across a wide range of tasks, we find that language models are mostly short-sighted: they usually track local next steps rather than holding a precise long-range plan. On easier tasks, they may sense a rough answer early, but on harder multi-step problems, the correct answer often becomes clear only near the end of reasoning. This helps reconcile why chain-of-thought can be both partly predictable and still necessary. We also show that a few critical reasoning moments can better reveal whether a model is uncertain, and that some easy cases can skip full chain-of-thought with little loss in accuracy. These findings improve our understanding of how AI systems reason and may help build models that think more efficiently and reliably.