Timezone: »

 
Poster
Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills
Yevgen Chebotar · Karol Hausman · Yao Lu · Ted Xiao · Dmitry Kalashnikov · Jacob Varley · Alexander Irpan · Benjamin Eysenbach · Ryan C Julian · Chelsea Finn · Sergey Levine

Tue Jul 20 09:00 PM -- 11:00 PM (PDT) @ Virtual
in Poster Session 2 »

We consider the problem of learning useful robotic skills from previously collected offline data without access to manually specified rewards or additional online exploration, a setting that is becoming increasingly important for scaling robot learning by reusing past robotic data. In particular, we propose the objective of learning a functional understanding of the environment by learning to reach any goal state in a given dataset. We employ goal-conditioned Q-learning with hindsight relabeling and develop several techniques that enable training in a particularly challenging offline setting. We find that our method can operate on high-dimensional camera images and learn a variety of skills on real robots that generalize to previously unseen scenes and objects. We also show that our method can learn to reach long-horizon goals across multiple episodes through goal chaining, and learn rich representations that can help with downstream tasks through pre-training or auxiliary objectives.

Keywords: Robotics
[ Slides] 

Author Information

Yevgen Chebotar (Google)
Karol Hausman (Google Brain)
Yao Lu (Google Research)
Ted Xiao (Google)
Dmitry Kalashnikov (Google Inc.)
Jacob Varley (Google)
Alexander Irpan (Google)
Benjamin Eysenbach (CMU, Google Brain)
Ryan C Julian (University of Southern California)
Chelsea Finn (Google Brain)
Sergey Levine (Google)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors