Beyond the Final Answer: Evaluating the Reasoning Trajectories of Tool-Augmented Agents

Although recent tool-augmented benchmarks involve complex requests, evaluation remains limited to answer matching, neglecting critical trajectory aspects like efficiency, hallucination, and adaptivity. The most straightforward method for evaluation is to compare an agent’s trajectory with the ground-truth, but annotating all valid ground-truth trajectories is prohibitively expensive. In this manner, we introduce TRACE, a reference free framework for the multi-dimensional evaluation of tool-augmented LLMs. By incorporating an evidence bank which accumulates knowledge from preceding steps, TRACE assesses an agent’s reasoning trajectory effectively. To validate our framework, we develop a new meta-evaluation dataset with diverse and flawed trajectories, each labeled with multi-faceted performance scores. Our results confirm that TRACE accurately evaluates complex trajectories even with small open source LLMs. Furthermore, we apply our method to evaluate the trajectories that agents produce while solving tool-augmented tasks, presenting previously unreported observations and their corresponding insights.

Current methods for evaluating AI agents often only check the final answer, ignoring the internal reasoning process. This fails to distinguish between a truly capable agent and one that is simply lucky, while also penalizing valid, alternative solution paths. We introduce TRACE, a reference-free framework that evaluates an AI agent's entire reasoning trajectory. It uses an "evidence bank" to store factual information gathered at each step, ensuring the AI's thoughts are logically grounded in reality. We specifically measure three key dimensions: efficiency, hallucination, and adaptivity to tool failures. Our analysis reveals that agents with similar accuracy can exhibit vastly different reasoning quality. By uncovering these hidden performance gaps, TRACE helps developers create AI assistants that are more efficient and reliable. This paves the way for trustworthy AI agents that can navigate complex, real-world tasks with greater precision.