Active Curriculum Refinement for Reinforcement Learning

In many RL domains, environments are linked by prerequisite relations—e.g., difficulty-increasing edits or parameter increments—which induce a directed acyclic curriculum graph (DAG). In practice, this structure is often exploited only implicitly, yet it can yield clear gains in training. We introduce PATH, a curriculum learning framework that performs active learning on the curriculum graph. PATH first expands coverage by sampling diverse curriculum paths, then reallocates training toward regions that remain unmastered. Experiments show that PATH leverages the graph structure to achieve strong robustness and generalization across diverse environments.

People often learn better when they start with easier practice problems and gradually move to harder ones. This idea is also used to train artificial intelligence agents. For example, if we want to teach a robot to walk, we might first let it walk on flat ground, then on small bumps, and later on stairs or gaps. But real training problems are often more complicated than a single easy-to-hard ladder. A robot may need to handle many different kinds of challenges at the same time: taller stairs, wider gaps, slippery ground, uneven surfaces, or obstacles placed in different locations. A game-playing AI may face harder maps, faster enemies, more objects, or different goals. When there are many ways for a task to become harder, it is no longer clear what the “right order” of training should be. This paper proposes PATH, a method for automatically building better training routes in these multi-dimensional difficulty settings. Instead of picking practice problems one at a time, PATH looks for connected routes through the environment space. Each route gradually changes the training problem along one or more known dimensions, such as making gaps wider or stairs taller. By identifying routes that are especially useful for learning, PATH can help the agent train more efficiently rather than spending equal effort on less informative tasks. This gives the agent a more organized and efficient way to train in practice.