The Geometric Reasoner: Manifold-Informed Latent Foresight Search for Long-Context Reasoning

Scaling test-time compute enhances long chain-of-thought (CoT) reasoning, yet existing approaches face a fundamental trade-off between computational cost and coverage quality: either incurring high training expense or yielding redundant trajectories. We introduce The Geometric Reasoner (TGR), a training-free framework that performs manifold-informed latent foresight search under strict memory bounds. At each chunk boundary, TGR scores candidate latent anchors via a lightweight look-ahead estimate combined with soft geometric regularizers that encourage smooth trajectories and diverse exploration. Chunk-wise KV cache resets keep memory linear in chunk length. On challenging math and code benchmarks, TGR improves robust trajectory coverage, measured by the area under the Pass@$k$ curve (AUC), by up to 13 points on Qwen3-8B, with negligible overhead of about 1.1--1.3$\times$.

Large language models can solve harder problems when they are allowed to think for longer, but extra thinking time is often wasted on repeated or very similar attempts. This is especially costly for long math, coding, or planning tasks, where the model must keep track of many steps. We introduce The Geometric Reasoner, a method that guides a model while it is generating an answer, without retraining it. The method periodically looks at several possible directions the model could take next and chooses one that is both promising and different from what has already been explored. This helps the model use its extra computation to cover more genuinely different solution paths instead of repeating itself. In experiments on math and code benchmarks, this led to more reliable answers with only a modest increase in computation. The work may help make long-form AI reasoning more dependable and efficient.