Learning to Self-Verify Makes Language Models Better Reasoners

Recent large language models (LLMs) achieve strong performance in generating promising reasoning paths for complex tasks. However, despite powerful generation ability, LLMs remain weak at verifying their own answers, revealing a persistent capability asymmetry between generation and self-verification. In this work, we conduct an in-depth investigation of this asymmetry throughout training evolution and show that, even on the same task, improving generation does not lead to corresponding improvements in self-verification. Interestingly, we find that the reverse direction of this asymmetry behaves differently: learning to self-verify can effectively improve generation performance, achieving accuracy comparable to standard generation training while yielding more efficient and effective reasoning traces. Building on this observation, we further explore integrating self-verification into generation training by formulating a multi-task reinforcement learning framework, where generation and self-verification are optimized as two independent but complementary objectives. Extensive experiments across benchmarks and models demonstrate performance gains over generation-only training in both generation and verification capabilities.

Large language models are becoming increasingly good at solving complex problems, but they still often struggle to judge whether their own answers are correct. In this paper, we study the relationship between a model’s ability to produce answers and its ability to check its own answers. We find that simply training a model to generate better solutions does not necessarily make it better at self-checking. Surprisingly, however, training a model to verify whether its own answers are correct can also make it a better problem solver. Based on this finding, we propose a training approach that teaches models both to solve problems and to verify their own solutions. Our experiments on mathematical reasoning tasks show that this approach improves problem-solving performance, strengthens self-verification ability, and often leads to shorter and more efficient reasoning. These results suggest that self-verification is not only useful for checking answers, but can also serve as an important way to improve how language models reason.