Timezone: »

 
Oral
Policy Optimization with Demonstrations
Bingyi Kang · Zequn Jie · Jiashi Feng

Fri Jul 13 02:50 AM -- 03:00 AM (PDT) @ A1
in Reinforcement Learning 15 »

Exploration remains a significant challenge to reinforcement learning methods, especially in environments where reward signals are sparse. Recent methods of learning from demonstrations have shown to be promising in overcoming exploration difficulties but typically require considerable high-quality demonstrations that are difficult to collect. We propose to effectively leverage available demonstrations to guide exploration through enforcing occupancy measure matching between the learned policy and current demonstrations, and develop a novel Policy Optimization from Demonstration (POfD) method. We show that POfD induces implicit dynamic reward shaping and brings provable benefits for policy improvement. Furthermore, it can be combined with policy gradient methods to produce state-of-the-art results, as demonstrated experimentally on a range of popular benchmark sparse-reward tasks, even when the demonstrations are few and imperfect.

[ PDF] 

Author Information

Bingyi Kang (National University of Singapore)
Zequn Jie (Tencent AI Lab)
Jiashi Feng (National University of Singapore)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors