Timezone: »

 
Oral
Curiosity-Bottleneck: Exploration By Distilling Task-Specific Novelty
Youngjin Kim · Daniel Nam · Hyunwoo Kim · Ji-Hoon Kim · Gunhee Kim

Tue Jun 11 03:05 PM -- 03:10 PM (PDT) @ Hall B
in Deep RL 2 »

Exploration based on state novelty has brought great success in challenging reinforcement learning problems with sparse rewards. However, existing novelty-based strategies become inefficient in real-world problems where observation contains not only task-dependent state novelty of our interest but also task-irrelevant information that should be ignored. We introduce an information-theoretic exploration strategy named Curiosity-Bottleneck that distills task-relevant information from observation. Based on the Information Bottleneck principle, our exploration bonus is quantified as the compressiveness of observation with respect to the learned representation of a compressive value network. With extensive experiments on static image classification, grid-world and three hard-exploration Atari games, we show that Curiosity-Bottleneck learns effective exploration strategy by robustly measuring the state novelty in distractive environment where state-of-the-art exploration methods often degenerate.

[ Slides]  [ Video] 

Author Information

Youngjin Kim (NALBI Inc.)
Daniel Nam (KC Machine Learning Lab)
Hyunwoo Kim (Seoul National University)
Ji-Hoon Kim (Naver Corp.)
Gunhee Kim (Seoul National University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors