DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Use

Recent work increasingly synthesizes agentic tasks for post-training tool-using LLMs, yet robust generalization under shifts in tasks and toolsets remains an open challenge. We trace this brittleness to insufficient diversity in synthesized training tasks. Scaling diversity is difficult because training requires tasks to remain executable and verifiable, while generalization demands diverse tool types, toolset combinations, and heterogeneous tool-use patterns. We propose DIVE, an evidence-driven recipe that inverts synthesis order, executing diverse real-world tools first and reverse-deriving tasks strictly entailed by the resulting traces, providing grounding by construction. DIVE scales structural diversity along two controllable axes, tool-pool coverage and per-task toolset variety, synthesizing 48k trajectories over 374 tools across five domains that cover 46,398 unique toolsets and 39,810 unique tool-call graphs. Training Qwen3-8B on DIVE data (48k SFT + 3.2k RL) improves by +22 average points across 9 OOD benchmarks and outperforms the strongest 8B baseline by +68%. Under a fixed budget, controlled scaling shows diversity scaling consistently outperforms quantity scaling, even with 4× less data.

Modern AI assistants are increasingly expected to use external tools—web search, calculators, databases, and so on—to accomplish real-world tasks. To train them, researchers create synthetic example tasks, but these tend to be narrow and repetitive. As a result, AI trained on them struggles when facing new tools or unfamiliar problem types. We introduce DIVE, a new way to generate diverse training tasks. Rather than writing tasks first and trying to solve them with tools, we flip the order: we run real tools first, observe what they produce, and then derive tasks from those observations. This guarantees every task is actually solvable. With DIVE, we generated tasks covering 373 tools across five domains (general knowledge, science, medicine, and finance). An AI trained on this data scored 22 points higher on average across 9 challenging evaluations. Notably, we found that diversity matters more than quantity—a small but varied dataset outperforms a much larger but repetitive one. This suggests that data diversity, rather than sheer volume, is the key to building AI assistants that can use new tools effectively in real-world scenarios.