Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning

Recent advances in large language model (LLM) have empowered autonomous agents to perform multi-turn interactions with tools and environments. However, scaling such agent training is limited by the lack of diverse and reliable environments.In this paper, we propose Agent World Model (AWM), a fully synthetic environment generation pipeline. Using this pipeline, we scale to 1,000 environments covering everyday scenarios, in which agents can interact with rich toolsets and obtain high-quality observations. Notably, these environments are code-driven and backed by databases, providing more reliable and consistent state transitions than environments simulated by LLMs. Moreover, they enable more efficient agent interaction compared with collecting trajectories from realistic environments. To demonstrate the effectiveness of this resource, we perform large-scale reinforcement learning for multi-turn tool-use agents. Thanks to the fully executable environments and accessible database states, we can also design reliable reward functions. Experiments on three benchmarks show that training exclusively in synthetic environments, rather than benchmark-specific ones, yields strong out-of-distribution generalization. The code is available at https://github.com/Snowflake-Labs/agent-world-model.

Modern AI assistants can do more than chat: they can take actions for you — booking flights, managing emails, updating records. To become reliable at this, an AI agent needs to practice in realistic settings, but real systems like banking websites or hospital records are expensive and risky to let an unfinished AI poke at thousands of times, and the handful of practice environments researchers have built by hand cover only a few narrow scenarios. We built Agent World Model (AWM), a system that automatically generates 1,000 entirely synthetic practice environments, each its own miniature application (a fake online store, a fake banking app, a fake hospital scheduler) with a working database and a set of tools the agent can call, all as real software. Since these worlds are actual programs rather than another AI "imagining" what should happen, they behave consistently and let us automatically grade what the agent did by inspecting the database. AI agents trained only on these synthetic worlds then performed strongly on three independent test suites built by other researchers for completely different applications, showing that practice transfers from synthetic to real and offering a cheaper, safer, and more scalable way to train the next generation of AI agents that act in the world.