Structure Enables Effective Self-Localization of Errors in LLMs

Self-correction in language models remains elusive. In this work, we explore whether language models can explicitly localize errors in incorrect reasoning, as a path toward building AI systems that can effectively correct themselves. We introduce a prompting method that structures reasoning as discrete, semantically coherent thought steps, and show that models can localize errors more reliably within this structure than in conventional, unstructured chain-of-thought reasoning. Motivated by how the human brain monitors errors at discrete decision points and resamples alternatives, we introduce Iterative Correction Sampling of Thoughts (Thought-ICS), a self-correction framework. Thought-ICS iteratively prompts the model to generate reasoning one discrete and complete thought at a time—where each thought represents a deliberate decision by the model—creating natural boundaries for precise error localization. Upon verification, the model localizes the first erroneous step, and the system backtracks to generate alternative reasoning from the last correct point. When asked to correct reasoning verified as incorrect by an oracle, Thought-ICS achieves 20-40% self-correction lift. In a completely autonomous setting without external verification, it outperforms contemporary self-correction baselines.

Self-correction in language models is usually done by critiquing an answer and regenerating the entire reasoning trace. We show a more targeted approach works better: localize the first erroneous step, then resample only from that point. The crux is self-localization. Rather than asking whether models can identify errors in arbitrary reasoning traces, we generate the reasoning so that localization is tractable in the first place. We frame reasoning as a Thought MDP, where at each step the model is asked to produce just the next complete thought, forming a chain of thoughts with self-determined boundaries. In this format, models reach a clean prefix, free of the error that derailed them, far more reliably than within unstructured chain-of-thought. And sampling new continuations from a clean prefix corrects substantially more often than from a flawed one. Under oracle verification, this yields a consistent correction lift. We evaluate how the abilities required for self-correction scale across models, finding that localization improves with size while self-verification does not, making it the bottleneck for fully autonomous self-correcting systems. We address this with confidence-based methods, building one that outperforms existing approaches that regenerate the whole trace. The broader takeaway: by structuring reasoning as a chain of semantically coherent thoughts, self-localization becomes tractable, enabling self-correction as targeted intervention rather than complete regeneration.