Talk
End-to-End Differentiable Adversarial Imitation Learning
Nir Baram · Oron Anschel · Itai Caspi · Shie Mannor

Mon Aug 7th 04:06 -- 04:24 PM @ C4.5

Generative Adversarial Networks (GANs) have been successfully applied to the problem of \emph{policy imitation} in a model-free setup. However, the computation graph of GANs, that include a stochastic policy as the generative model, is no longer differentiable end-to-end, which requires the use of high-variance gradient estimation. In this paper, we introduce the Model-based Generative Adversarial Imitation Learning (MGAIL) algorithm. We show how to use a forward model to make the computation fully differentiable, which enables training policies using the exact gradient of the discriminator. The resulting algorithm trains competent policies using relatively fewer expert samples and interactions with the environment. We test it on both discrete and continuous action domains and report results that surpass the state-of-the-art.

Author Information

Nir Baram (Technion - Israel Institute of Technology)
Oron Anschel (Technion)
Itai Caspi (Technion)
Shie Mannor (Technion)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors