Think in Latent, Explain in Language: Self-Explainable Latent Reasoning

Latent reasoning has emerged as a powerful alternative to text-based Chain-of-Thought (CoT), offering significant gains in computational efficiency by compressing verbose reasoning into compact embeddings. However, compressing reasoning into the latent space renders the thinking opaque, hindering its interpretability. Current methods present a stark trade-off: they either function as unexplainable “black boxes” (e.g., Coconut), where the latent reasoning is not human-readable, or rely on separate post-hoc decoders for explainability (e.g., Heima), introducing architectural overhead and decoupling the explanation from the actual reasoning process. In this work, we present a unified framework for Self-Explainable Latent Reasoning (SELR) that trains a single model to perform efficient and inherently explainable latent reasoning. Our core contribution is a novel multi-task training objective that optimizes for two goals simultaneously: (1) an Answer Loss that optimizes the latent reasoning trajectory to produce accurate final answers, and (2) a CoT Loss that explicitly trains the same model to decode its own latent representations back into human-understandable reasoning steps. This design ensures that generated latent representations are both task-effective and semantically interpretable, eliminating the need for external decoders. We validate the effectiveness of SELR on both Large Language Models (LLMs) and Vision-Language Models (VLMs), demonstrating that SELR achieves superior token efficiency and accuracy compared to baselines, while uniquely providing self-contained explainability without auxiliary models.

Modern AI assistants often "think out loud" before answering, writing long chains of reasoning steps in plain text. This makes them slow and expensive to run, but it has one big advantage: we can read their reasoning and check it. Recent work speeds these models up by letting them think in compressed internal vectors instead of words, but this turns the reasoning process into a black box that we cannot understand. We asked whether a model could enjoy the speed of internal latent thinking while still being able to explain itself in plain language. Our method, SELR, trains a single model to do both at once: it reasons through compact internal vectors, and the very same model can decode each of those vectors back into a human-readable explanation on demand. On vision and math benchmarks, SELR matches or beats standard text-based reasoning while generating about 70% fewer tokens and running up to 8.7x faster. Crucially, the explanations it produces provides a window into how the model actually reached its conclusion. This brings us closer to AI that is both efficient and explainable.