AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning

Agentic reinforcement learning has advanced large language models (LLMs) to reason through long chain-of-thought trajectories while interleaving external tool use. Existing approaches assume a fixed inventory of tools, which limits the adaptability of LLM agents to new or evolving toolsets. We present AutoTool, a training framework that equips LLM agents with dynamic tool-selection capabilities throughout their reasoning trajectories. AutoTool employs a dual-phase optimization pipeline: (i) SFT and RL-based trajectory stabilization for coherent reasoning, and (ii) KL-regularized Plackett–Luce Ranking to refine consistent multi-step tool selection. We further build a 200k dataset with explicit tool-selection rationales across 1,000+ tools and 100+ tasks spanning mathematics, science, code generation, and multimodal reasoning. Across ten diverse benchmarks, we train two base models, Qwen3-8B and Qwen2.5-VL-7B, with AutoTool. With fewer parameters, AutoTool consistently outperforms advanced LLM agents and tool-integration methods, yielding average gains of 6.4\% in math & science reasoning, 4.5\% in search-based QA, 7.7\% in code generation, and 6.9\% in multimodal understanding. In addition, AutoTool exhibits stronger generalization by dynamically leveraging unseen tools from evolving toolsets during inference.

Large language models are increasingly used as agents that can reason step by step and call external tools such as search engines, code interpreters, or vision tools. However, most existing agent systems assume a fixed set of tools, which makes them less flexible when new tools are added or when the task requires choosing among many possible tools. We introduce AutoTool, a training framework that teaches language-model agents to dynamically select and use the right tools during reasoning. AutoTool first stabilizes the model’s reasoning trajectory through supervised fine-tuning and reinforcement learning, then further improves tool selection using a ranking-based optimization method that encourages better tool choices over weaker alternatives. Across math, science, search, code, and multimodal tasks, AutoTool improves performance over standard agent-training and tool-use baselines. It also generalizes to previously unseen tools, suggesting a path toward more adaptable AI agents that can operate in evolving tool environments.