Timezone: »
We consider the problem of imitation learning from a finite set of expert trajectories, without access to reinforcement signals. The classical approach of extracting the expert's reward function via inverse reinforcement learning, followed by reinforcement learning is indirect and may be computationally expensive. Recent methods based on generative adversarial networks or generative moment matching formulate the task as distribution matching between the expert policy and the learned policy. However, training via distribution matching could be unstable. We propose a new framework for imitation learning based on estimating the support of the expert policy to compute a fixed reward function from the expert trajectories. This allows us to re-frame imitation learning within the standard reinforcement learning setting. We demonstrate the efficacy of our reward function on both discrete and continuous domains. The policies learned using different reinforcement learning methods with the proposed reward function achieve comparable or better performance than other imitation learning methods.
Author Information
Ruohan Wang (Imperial College London)
Carlo Ciliberto (Imperial College London)
Pierluigi Vito Amadori (Imperial College London)
Yiannis Demiris (Imperial College London)
Related Events (a corresponding poster, oral, or spotlight)
-
2019 Poster: Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation »
Wed. Jun 12th 01:30 -- 04:00 AM Room Pacific Ballroom
More from the Same Authors
-
2022 Poster: Measuring dissimilarity with diffeomorphism invariance »
Théophile Cantelobre · Carlo Ciliberto · Benjamin Guedj · Alessandro Rudi -
2022 Spotlight: Measuring dissimilarity with diffeomorphism invariance »
Théophile Cantelobre · Carlo Ciliberto · Benjamin Guedj · Alessandro Rudi -
2022 Poster: Distribution Regression with Sliced Wasserstein Kernels »
Dimitri Marie Meunier · Carlo Ciliberto · Massimiliano Pontil -
2022 Spotlight: Distribution Regression with Sliced Wasserstein Kernels »
Dimitri Marie Meunier · Carlo Ciliberto · Massimiliano Pontil -
2019 Poster: Learning-to-Learn Stochastic Gradient Descent with Biased Regularization »
Giulia Denevi · Carlo Ciliberto · Riccardo Grazzi · Massimiliano Pontil -
2019 Oral: Learning-to-Learn Stochastic Gradient Descent with Biased Regularization »
Giulia Denevi · Carlo Ciliberto · Riccardo Grazzi · Massimiliano Pontil -
2019 Poster: Leveraging Low-Rank Relations Between Surrogate Tasks in Structured Prediction »
Giulia Luise · Dimitrios Stamos · Massimiliano Pontil · Carlo Ciliberto -
2019 Oral: Leveraging Low-Rank Relations Between Surrogate Tasks in Structured Prediction »
Giulia Luise · Dimitrios Stamos · Massimiliano Pontil · Carlo Ciliberto