Characterizing, Evaluating, and Optimizing Complex Reasoning

Large Reasoning Models (LRMs) increasingly rely on reasoning traces with complex internal structures. However, existing work lacks a unified answer to three fundamental questions: (1) what defines high-quality reasoning, (2) how to reliably evaluate long, implicitly structured reasoning traces, and (3) how to use such evaluation signals for reasoning optimization. To address these challenges, we provide a unified perspective. (1) We introduce the ME$^2$ principle to characterize reasoning quality along macro- and micro-level concerning efficiency and effectiveness. (2) Built on this principle, we model reasoning traces as directed acyclic graphs (DAGs) and develop a DAG-based pairwise evaluation method, capturing complex reasoning structures. (3) Based on this method, we construct the TRM-Preference dataset and train a Thinking Reward Model (TRM) to evaluate reasoning quality at scale. Experiments show that thinking rewards serve as an effective optimization signal. At test time, selecting better reasoning leads to better outcomes (up to 19.3\% gain), and during RL training, thinking rewards enhance reasoning and performance (up to 3.9\% gain) across diverse tasks. Code and data are available at https://github.com/Simplified-Reasoning/TRM.

Large reasoning models often solve difficult problems by writing long chains of intermediate thoughts before giving a final answer. These traces can help, but they can also be wasteful or misleading: a model may repeat itself, follow irrelevant directions, or hide small mistakes inside a long explanation. This paper studies how to judge and improve the quality of such reasoning, separately from whether the final answer is correct. We describe good reasoning in terms of both structure and content. It should stay organized, avoid unnecessary detours, make valid local steps, and use concise steps that move the solution forward. To evaluate this, we turn each reasoning trace into a directed acyclic graph showing how its steps progress, branch, and merge, then compare pairs of traces using these criteria. From these comparisons, we build a dataset and train a Thinking Reward Model to score reasoning quality automatically. Using this score to choose better traces when a model answers questions improves final accuracy by up to 19.3%, and using it during training improves performance by up to 3.9%. This suggests that AI systems can become more reliable and efficient by rewarding not only correct answers, but also better ways of reaching them.