Timezone: »

 
Poster
Growing Action Spaces
Gregory Farquhar · Laura Gustafson · Zeming Lin · Shimon Whiteson · Nicolas Usunier · Gabriel Synnaeve

Tue Jul 14 01:00 PM -- 01:45 PM & Wed Jul 15 02:00 AM -- 02:45 AM (PDT) @
in Poster Session 7 »

In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress. In this work, we use a curriculum of progressively growing action spaces to accelerate learning. We assume the environment is out of our control, but that the agent may set an internal curriculum by initially restricting its action space. Our approach uses off-policy reinforcement learning to estimate optimal value functions for multiple action spaces simultaneously and efficiently transfers data, value estimates, and state representations from restricted action spaces to the full task. We show the efficacy of our approach in proof-of-concept control tasks and on challenging large-scale StarCraft micromanagement tasks with large, multi-agent action spaces.

Keywords: Deep Reinforcement Learning · Multiagent Learning · Reinforcement Learning · Reinforcement Learning - Deep RL

Author Information

Gregory Farquhar (Deepmind)
Laura Gustafson (Facebook AI Research)
Zeming Lin (New York University)
Shimon Whiteson (Oxford University)
Nicolas Usunier (Facebook AI Research)
Gabriel Synnaeve (Facebook AI Research)

More from the Same Authors