Verifying Meta-Awareness via Predictive Rewards in Reasoning Models

Recent research on reasoning models explores the meta-awareness of language models, including their ability to determine optimal thinking duration, recognize knowledge boundaries, and structure concept-level thinking. While current large reasoning models depend solely on answer-based verification, we show that adding meta-awareness objectives leads to significant performance gains over models without such meta-knowledge. **MAPR** utilizes a self-generated task of predicting rollout statistics - specifically length, pass-rate, and concepts used - allowing for verification against the actual statistics. Furthermore, by leveraging this self-predictive capability, the model can regulate its reasoning behavior by i) filtering out trivial or unsolvable prompts, ii) reducing lengthy generations that tend to be incorrect, and iii) generating hints relevant to the problem. The results are promising: **MAPR** yields significant improvements in both accuracy and training efficiency on various reasoning benchmarks. More specifically, our method can speed up GRPO training by over 1.28$\times$ to reach the same performance, and achieve 83.18\% gain in accuracy on AIME25, and a 16.45\% average gain over six mathematics benchmarks.

(1) Large reasoning models can solve complex tasks, but they struggle to recognize the boundaries of their own knowledge and uncertainty. We asked whether a model could generate useful learning signals not only from final answers, but also from predicting how it succeed. (2) To study this, we designed a parallel reasoning pipeline where the model simultaneously answers a question and predicts meta-information about its own reasoning process, including estimated solving success rate, concept usage keywords, and expected solution length. These self-assessments form an auxiliary reward pathway that operates alongside standard answer matching. (3) Our findings suggest that meta-prediction can improve model performance even without additional supervision beyond question-answer pairs. By learning to estimate its own reasoning quality and knowledge boundaries, the model gains a richer training signal that complements direct correctness rewards. This opens a path toward reasoning systems that are not only stronger problem solvers, but also better calibrated and more self-aware during learning.