Timezone: »

 
Poster
Jump-Start Reinforcement Learning
Ikechukwu Uchendu · Ted Xiao · Yao Lu · Banghua Zhu · Mengyuan Yan · Joséphine Simon · Matthew Bennice · Chuyuan Fu · Cong Ma · Jiantao Jiao · Sergey Levine · Karol Hausman

Thu Jul 27 04:30 PM -- 06:00 PM (PDT) @ Exhibit Hall 1 #201
in Poster Session 6 »

Reinforcement learning (RL) provides a theoretical framework for continuously improving an agent's behavior via trial and error. However, efficiently learning policies from scratch can be very difficult, particularly for tasks that present exploration challenges. In such settings, it might be desirable to initialize RL with an existing policy, offline data, or demonstrations. However, naively performing such initialization in RL often works poorly, especially for value-based methods. In this paper, we present a meta algorithm that can use offline data, demonstrations, or a pre-existing policy to initialize an RL policy, and is compatible with any RL approach. In particular, we propose Jump-Start Reinforcement Learning (JSRL), an algorithm that employs two policies to solve tasks: a guide-policy, and an exploration-policy. By using the guide-policy to form a curriculum of starting states for the exploration-policy, we are able to efficiently improve performance on a set of simulated robotic tasks. We show via experiments that it is able to significantly outperform existing imitation and reinforcement learning algorithms, particularly in the small-data regime. In addition, we provide an upper bound on the sample complexity of JSRL and show that with the help of a guide-policy, one can improve the sample complexity for non-optimism exploration methods from exponential in horizon to polynomial.

[ PDF]  [ Poster] 

Author Information

Ikechukwu Uchendu (Harvard University)
Ted Xiao (Google)
Yao Lu (Google Deepmind)
Banghua Zhu (University of California, Berkeley)
Mengyuan Yan (Google)
Joséphine Simon (Georgia Institute of Technology)
Matthew Bennice (Google)
Chuyuan Fu (Google)
Cong Ma (University of Chicago)
Jiantao Jiao (University of California, Berkeley)
Sergey Levine (University of Washington)
Karol Hausman (Google Brain)

More from the Same Authors