Timezone: »

 
Poster
AR-DAE: Towards Unbiased Neural Entropy Gradient Estimation
Jae Hyun Lim · Aaron Courville · Christopher Pal · Chin-Wei Huang

Thu Jul 16 08:00 AM -- 08:45 AM & Thu Jul 16 07:00 PM -- 07:45 PM (PDT) @
in Poster Session 43 »

Entropy is ubiquitous in machine learning, but it is in general intractable to compute the entropy of the distribution of an arbitrary continuous random variable. In this paper, we propose the amortized residual denoising autoencoder (AR-DAE) to approximate the gradient of the log density function, which can be used to estimate the gradient of entropy. Amortization allows us to significantly reduce the error of the gradient approximator by approaching asymptotic optimality of a regular DAE, in which case the estimation is in theory unbiased. We conduct theoretical and experimental analyses on the approximation error of the proposed method, as well as extensive studies on heuristics to ensure its robustness. Finally, using the proposed gradient approximator to estimate the gradient of entropy, we demonstrate state-of-the-art performance on density estimation with variational autoencoders and continuous control with soft actor-critic.

Keywords: Bayesian Deep Learning · Information Theory and Estimation · Reinforcement Learning · Autoencoders · Deep Learning - Generative Models and Autoencoders

Author Information

Jae Hyun Lim (Mila)

I am a graduate student at Mila, advised by Chris Pal. My research interest is about approximate inference in deep generative models and deep reinforcement learning.

Aaron Courville (Université de Montréal)
Christopher Pal (École Polytechnique de Montréal)
Chin-Wei Huang (MILA)

More from the Same Authors