THE CONFERENCE
FOR AUTHORS
FOR PARTICIPANTS

Timezone: »

 
Poster
Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Prediction
Wen Sun · Arun Venkatraman · Geoff Gordon · Byron Boots · Drew Bagnell

Wed Aug 09 01:30 AM -- 05:00 AM (PDT) @ Gallery #121
in Posters Wed »

Recently, researchers have demonstrated state-of-the-art performance on sequential prediction problems using deep neural networks and Reinforcement Learning (RL). For some of these problems, oracles that can demonstrate good performance may be available during training, but are not used by plain RL methods. To take advantage of this extra information, we propose AggreVaTeD, an extension of the Imitation Learning (IL) approach of Ross & Bagnell (2014). AggreVaTeD allows us to use expressive differentiable policy representations such as deep networks, while leveraging training-time oracles to achieve faster and more accurate solutions with less training data. Specifically, we present two gradient procedures that can learn neural network policies for several problems, including a sequential prediction task and several high-dimensional robotics control problems. We also provide a comprehensive theoretical study of IL that demonstrates that we can expect up to exponentially-lower sample complexity for learning with AggreVaTeD than with plain RL algorithms. Our results and theory indicate that IL (and AggreVaTeD in particular) can be a more effective strategy for sequential prediction than plain RL.

Author Information

Wen Sun (Carnegie Mellon University)
Arun Venkatraman (Carnegie Mellon University)
Geoff Gordon (Carnegie Mellon University)
Byron Boots (Georgia Tech)
Drew Bagnell (Carnegie Mellon University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors