Less Is More: Fast and Accurate Reasoning with Cross-Head Unified Sparse Attention

Large reasoning models achieve strong performance through test-time scaling, but this incurs substantial computational overhead due to long decoding from short prompts. While sparse attention can reduce latency and memory usage, existing methods often degrade reasoning accuracy because selection errors accumulate over long generation horizons, or require costly retraining. We introduce LessIsMore, a training-free sparse attention mechanism for long-horizon reasoning. Our key insight is that token importance in reasoning is global and stable: critical tokens are largely shared across attention heads and remain stable over decoding steps. Guided by this structure, LessIsMore enforces cross-head unified token selection and preserves recent context via a stable recency window, yielding a globally consistent token set that can be reused across layers. Across multiple model families and challenging reasoning benchmarks, LessIsMore matches or improves accuracy while attending to substantially fewer tokens. With kernel-level optimizations, LessIsMore achieves up to $1.6\times$ end-to-end decoding speedup and up to $1.72\times$ faster sparse attention computation, with additional long-context results demonstrating the generality of our approach.

Modern AI systems that "think out loud" to solve math, science, and coding problems are powerful but slow and expensive, because each new word they generate forces them to "re-read" everything they have already written in attention mechanism. Our method, LessIsMore, exploits the observation that the words truly worth re-reading are largely shared across the model's many internal mechanisms and change only slowly, so it commits to one stable, shared set of important words instead of many inconsistent ones. The result is a training-free technique that runs reasoning models up to 1.6× faster while matching or beating their original accuracy on math, science, and coding benchmarks — making today's most capable reasoning AI noticeably cheaper and more energy-efficient to deploy.