Invited Talk: Feedback in Imitation Learning: Confusion on Causality and Covariate Shift (Arun Venkatraman & Sanjiban Choudhury)

Video: https://slideslive.com/38930758/feedback-in-imitation-learning

Abstract: Imitation learning practitioners have often noted that adding previous actions as a feature leads to a dramatic divergence between “held out” error and performance of the learner in situ. Interactive approaches (Ross et al., 2011; de Haan et al.,2019) can provably address this divergence but require repeated querying of a demonstrator. Recent work identifies this divergence as stemming from a “causal confound” (Pearl et al., 2016) in predicting the current action, and seek to ablate away past actions using tools from causal inference.

In this work, we first conclude that neither the stated model nor the experimental setups exhibit any causal confounding, and thus this cannot explain the empirical observations. We note that in these settings of feedback between decisions and features, the learner comes to rely on features that are strongly predictive of decisions but are also subject to strong covariate shift. Our work demonstrates a broad class of problems where this shift can be mitigated, both theoretically and practically, by taking advantage of a simulator but without any further querying of expert demonstration. We evaluate our approach on several benchmark control domains and show that it outperforms other baselines that use only such cached demonstrations.

Bio: Arun Venkatraman is a founding engineer at Aurora, the company delivering self-driving technology safely, quickly, and broadly. Arun graduated with a BS with Honors from the California Institute of Technology and completed his PhD, Training Strategies for Time Series: Learning for Prediction, Filtering, and Reinforcement Learning, at the Robotics Institute at Carnegie Mellon University co-advised by Dr. Drew Bagnell and Dr. Martial Hebert. During his time at CMU and NREC, Arun worked on a variety of robotics applications and received a best paper award at Robotics Science and Systems 2015 for work on autonomy assisted teleoperation via a brain-computer interface. At Aurora, Arun leads the Motion Planning Machine Learning team, bringing together the best in machine learning with the best practices in robotics development to develop the Aurora Driver.

Sanjiban Choudhury is a research engineer at Aurora, where he works with the best to solve self-driving at scale. He focuses on theory and algorithms at the intersection of machine learning and motion planning. Much of his research has been deployed on real-world robotic systems - full-scale helicopters, self-driving cars and mobile manipulators. He has a PhD from The Robotics Institute at Carnegie Mellon University, where he was advised by Sebastian Scherer. His thesis showed how robots can learn from prior experience to speed up online planning. He was a Postdoctoral fellow at the University of Washington, CSE where he worked with Sidd Srinivasa. He is the recipient of best paper awards at AHS 2014 and ICAPS 2019, winner of the 2018 Howard Hughes award and a 2013 Siebel’s Scholar.