On Effectiveness and Efficiency of Agentic Tool-calling and RL Training

Tool-calling is a central component of modern large language model (LLM) agents, equipping them with skills beyond their parametric knowledge. This paper studies tool-calling along two complementary axes: effectiveness, i.e., how this capability is measured, and efficiency, i.e., how it is learned. On effectiveness, we systematically analyze tool-calling evaluation pipelines and show that results can be highly sensitive to seemingly minor, often undocumented implementation choices including the random seed, system prompt, multi-turn template construction, and how prior interaction/reasoning history is carried forward. These choices can lead to substantial differences in reported performance, especially in multi-turn settings where without rigorous standardization, leaderboard rankings are unreliable. On efficiency, we examine standard reinforcement learning (RL) for tool-calling and identify two sources of computational waste: (i) during rollouts, many prompts produce no learning signal, and (ii) during policy updates, optimization incurs high computational cost. Guided by these findings, we introduce two techniques that accelerate RL-based tool-calling training, achieving substantial wall-clock speedup without degrading performance.

AI assistants are no longer limited to only writing text. Many of them can now use external tools, such as search engines, calculators, databases, or booking systems, to help users complete more complex tasks. In this work, we study two practical questions: how reliably we evaluate them, and how efficiently we train them. First, we ask whether current evaluations really measure how good these models are. We find that the benchmark results can be highly sensitive to seemingly small and often undocumented implementation choices, such as the random seed, system prompt, how multi-turn conversations are formatted, and how prior interaction/reasoning history is carried forward. This means that current comparisons between models or training methods can be misleading if these details are not carefully standardized or reported. We then study how these models are trained. We find much of the computation is wasted in the current setting, e.g., on training examples that no longer help the model learn. We propose simple ways to skip less useful examples and focus updates on more useful responses. Overall, our work helps the tool-calling community evaluate AI assistants more fairly and train them more efficiently.