Dreaming in Code for Curriculum Learning in Open-Ended Worlds

Open-ended learning frames intelligence as emerging from continual interaction with an ever-expanding space of environments. While recent advances have utilized foundation models to programmatically generate diverse environments, these approaches often focus on discovering isolated behaviors rather than orchestrating sustained progression. In complex open-ended worlds, the large combinatorial space of possible challenges makes it difficult for agents to discover sequences of experiences that remain consistently learnable. To address this, we propose Dreaming in Code (DiCode), a framework in which foundation models synthesize executable environment code to scaffold learning toward increasing competence. In DiCode, “dreaming” takes the form of materializing code-level variations of the world. We instantiate DiCode in Craftax, a challenging open-ended benchmark characterized by rich mechanics and long-horizon progression. Empirically, DiCode enables agents to acquire long-horizon skills, achieving a 17% improvement in mean return over the strongest baseline and non-zero success on late-game combat tasks where prior methods fail. Our results suggest that code-level environment design provides a practical mechanism for curriculum control, enabling the construction of intermediate environments that bridge competence gaps in open-ended worlds.

AI agents that learn by trial and error in complex, open-ended simulated worlds tend to make rapid early progress and then stall. The bottleneck is not the learner but its diet of experience: the agent rarely encounters situations that are hard enough to teach new skills, yet learnable enough to make progress. Our method, Dreaming in Code (DiCode), addresses this by letting a foundation model – a general-purpose AI – dream up new training worlds for the agent and generate the code that brings them to life. Concretely, the model writes the code that defines a new world, the simulator brings it to life, and the agent learns from the experience of interacting with it. The model then reviews what the agent currently masters and where it fails, designing each new world to sit just beyond the agent's current ability – much like a good teacher choosing the next exercise for a student. In a complex Minecraft-like game, agents trained with DiCode acquire late-stage skills that remain out of reach for existing methods. More broadly, our results demonstrate a new role for foundation models in open-ended learning: not as the source of an agent's knowledge, but as generators of the grounded experience it learns from.