Timezone: »

 
Spotlight
MetaCURE: Meta Reinforcement Learning with Empowerment-Driven Exploration
Jin Zhang · Jianhao Wang · Hao Hu · Tong Chen · Yingfeng Chen · Changjie Fan · Chongjie Zhang

Thu Jul 22 06:30 PM -- 06:35 PM (PDT) @
in Multi-task Learning 2 »

Meta reinforcement learning (meta-RL) extracts knowledge from previous tasks and achieves fast adaptation to new tasks. Despite recent progress, efficient exploration in meta-RL remains a key challenge in sparse-reward tasks, as it requires quickly finding informative task-relevant experiences in both meta-training and adaptation. To address this challenge, we explicitly model an exploration policy learning problem for meta-RL, which is separated from exploitation policy learning, and introduce a novel empowerment-driven exploration objective, which aims to maximize information gain for task identification. We derive a corresponding intrinsic reward and develop a new off-policy meta-RL framework, which efficiently learns separate context-aware exploration and exploitation policies by sharing the knowledge of task inference. Experimental evaluation shows that our meta-RL method significantly outperforms state-of-the-art baselines on various sparse-reward MuJoCo locomotion tasks and more complex sparse-reward Meta-World tasks.

Keywords: Algorithms · Multitask, Transfer, and Meta Learning

Author Information

Jin Zhang (Tsinghua University)
Jianhao Wang (Tsinghua University)
Hao Hu (Tsinghua University)
Tong Chen (Tsinghua University)
Yingfeng Chen (NetEase Fuxi AI Lab)
Changjie Fan (NetEase Fuxi AI Lab)
Chongjie Zhang (Tsinghua University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors