Poster
Amortized Population Gibbs Samplers with Neural Sufficient Statistics
Hao Wu · Heiko Zimmermann · Eli Sennesh · Tuan Anh Le · Jan-Willem van de Meent
Keywords: [ Approximate Inference ] [ Bayesian Methods ] [ Deep Generative Models ] [ Unsupervised Learning ] [ Probabilistic Inference - Approximate, Monte Carlo, and Spectral Methods ]
We develop amortized population Gibbs (APG) samplers, a class of scalable methods that frame structured variational inference as adaptive importance sampling. APG samplers construct high-dimensional proposals by iterating over updates to lower-dimensional blocks of variables. We train each conditional proposal by minimizing the inclusive KL divergence with respect to the conditional posterior. To appropriately account for the size of the input data, we develop a new parameterization in terms of neural sufficient statistics. Experiments show that APG samplers can be used to train highly-structured deep generative models in an unsupervised manner, and achieve substantial improvements in inference accuracy relative to standard autoencoding variational methods.