TRACE: Toulmin-based Reasoning Assessment through Constructive Elements for LLM CoT Evaluation

Evaluating open-ended outputs from large language models (LLMs) remains challenging due to the absence of ground truth. Existing metrics rely on final-answer accuracy or surface-level statistics, leaving the reasoning process itself unexamined. We introduce TRACE (Toulmin-based Reasoning Assessment through Constructive Elements), a metric that analyzes Chain-of-Thought (CoT) reasoning processes. Rather than judging outcomes, TRACE inspects how arguments are constructed by integrating Toulmin's argumentation theory with Flavell's metacognitive framework to assess reasoning structure. Experiments on 26.3K QA samples across 7 reasoning models show strong correlation with benchmark accuracy (r=0.74). Furthermore, TRACE is effective as a reinforcement learning reward signal, outperforming accuracy-only baselines. Together, these results indicate that logically sound reasoning leads to higher-quality answers. TRACE thus serves as a complementary metric for evaluating open-ended outputs. Code is available at https://github.com/hyyangkisti/trace.

Modern AI systems like ChatGPT often solve hard problems by "thinking out loud." They write out their reasoning step by step before giving an answer. But how do we know if that reasoning is actually sound? Maybe the AI just stumbled onto the right answer by luck. Today's evaluations mostly check the final answer. They ignore the thinking that led to it. We developed TRACE, a tool that inspects the reasoning itself. It borrows ideas from the study of how humans build convincing arguments. TRACE breaks each step of an AI's thought process into building blocks like claims, supporting evidence, and self-checks. It then measures whether these blocks connect together logically. We tested this across many AI systems and tasks. AI systems that reasoned more logically gave more accurate answers. TRACE can also be used during training to teach AI systems to reason more soundly. It offers a lightweight way to diagnose and improve how machines think.