Skip to yearly menu bar Skip to main content


Talk

End-to-End Differentiable Adversarial Imitation Learning

Nir Baram · Oron Anschel · Itai Caspi · Shie Mannor

C4.5

Abstract:

Generative Adversarial Networks (GANs) have been successfully applied to the problem of \emph{policy imitation} in a model-free setup. However, the computation graph of GANs, that include a stochastic policy as the generative model, is no longer differentiable end-to-end, which requires the use of high-variance gradient estimation. In this paper, we introduce the Model-based Generative Adversarial Imitation Learning (MGAIL) algorithm. We show how to use a forward model to make the computation fully differentiable, which enables training policies using the exact gradient of the discriminator. The resulting algorithm trains competent policies using relatively fewer expert samples and interactions with the environment. We test it on both discrete and continuous action domains and report results that surpass the state-of-the-art.

Live content is unavailable. Log in and register to view live content