Amortized Population Gibbs Samplers with Neural Sufficient Statistics

Hao Wu · Heiko Zimmermann · Eli Sennesh · Tuan Anh Le · Jan-Willem van de Meent

Keywords: [ Approximate Inference ] [ Bayesian Methods ] [ Deep Generative Models ] [ Unsupervised Learning ] [ Probabilistic Inference - Approximate, Monte Carlo, and Spectral Methods ]

Abstract [ Join Zoom ]
Please do not share or post zoom links


We develop amortized population Gibbs (APG) samplers, a class of scalable methods that frame structured variational inference as adaptive importance sampling. APG samplers construct high-dimensional proposals by iterating over updates to lower-dimensional blocks of variables. We train each conditional proposal by minimizing the inclusive KL divergence with respect to the conditional posterior. To appropriately account for the size of the input data, we develop a new parameterization in terms of neural sufficient statistics. Experiments show that APG samplers can be used to train highly-structured deep generative models in an unsupervised manner, and achieve substantial improvements in inference accuracy relative to standard autoencoding variational methods.

Live content is unavailable. Are you logged in?