Timezone: »

 
Oral
EMI: Exploration with Mutual Information
Hyoungseok Kim · Jaekyeom Kim · Yeonwoo Jeong · Sergey Levine · Hyun Oh Song

Tue Jun 11 02:40 PM -- 03:00 PM (PDT) @ Hall B
in Deep RL 2 »

Reinforcement learning algorithms struggle when the reward signal is very sparse. In these cases, naive random exploration methods essentially rely on a random walk to stumble onto a rewarding state. Recent works utilize intrinsic motivation to guide the exploration via generative models, predictive forward models, or discriminative modeling of novelty. We propose EMI, which is an exploration method that constructs embedding representation of states and actions that does not rely on generative decoding of the full observation but extracts predictive signals that can be used to guide exploration based on forward prediction in the representation space. Our experiments show that the proposed method significantly outperforms a number of existing exploration methods on challenging locomotion task with continuous control and on image-based exploration tasks with discrete actions on Atari.

[ Video] 

Author Information

Hyoungseok Kim (Seoul National University)
Jaekyeom Kim (Seoul National University)
Yeonwoo Jeong (Seoul National University)
Sergey Levine (UC Berkeley)
Sergey Levine

Sergey Levine received a BS and MS in Computer Science from Stanford University in 2009, and a Ph.D. in Computer Science from Stanford University in 2014. He joined the faculty of the Department of Electrical Engineering and Computer Sciences at UC Berkeley in fall 2016. His work focuses on machine learning for decision making and control, with an emphasis on deep learning and reinforcement learning algorithms. Applications of his work include autonomous robots and vehicles, as well as computer vision and graphics. His research includes developing algorithms for end-to-end training of deep neural network policies that combine perception and control, scalable algorithms for inverse reinforcement learning, deep reinforcement learning algorithms, and more.

Hyun Oh Song (Seoul National University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors