Timezone: »
Variational auto-encoders (VAE) are popular deep latent variable models which are trained by maximizing an Evidence Lower Bound (ELBO). To obtain tighter ELBO and hence better variational approximations, it has been proposed to use importance sampling to get a lower variance estimate of the evidence. However, importance sampling is known to perform poorly in high dimensions. While it has been suggested many times in the literature to use more sophisticated algorithms such as Annealed Importance Sampling (AIS) and its Sequential Importance Sampling (SIS) extensions, the potential benefits brought by these advanced techniques have never been realized for VAE: the AIS estimate cannot be easily differentiated, while SIS requires the specification of carefully chosen backward Markov kernels. In this paper, we address both issues and demonstrate the performance of the resulting Monte Carlo VAEs on a variety of applications.
Author Information
Achille Thin (Ecole polytechnique)
Nikita Kotelevskii (Skolkovo Institute of Science and Technology)
Arnaud Doucet (Oxford University)
Alain Durmus (ENS Paris Saclay)
Eric Moulines (Ecole Polytechnique)
Maxim Panov (Skolkovo Institute of Science and Technology)
Related Events (a corresponding poster, oral, or spotlight)
-
2021 Spotlight: Monte Carlo Variational Auto-Encoders »
Fri. Jul 23rd 01:45 -- 01:50 AM Room
More from the Same Authors
-
2021 : Model-Free Approach to Evaluate Reinforcement Learning Algorithms »
Denis Belomestny · Ilya Levin · Eric Moulines · Alexey Naumov · Sergey Samsonov · Veronika Zorina -
2022 : Riemannian Diffusion Schr\"odinger Bridge »
James Thornton · Valentin De Bortoli · Michael Hutchinson · Emile Mathieu · Yee Whye Teh · Arnaud Doucet -
2022 : Towards OOD Detection in Graph Classification from Uncertainty Estimation Perspective »
Gleb Bazhenov · Sergey Ivanov · Maxim Panov · Alexey Zaytsev · Evgeny Burnaev -
2023 : Diffusion Generative Inverse Design »
Marin Vlastelica · Tatiana Lopez-Guevara · Kelsey Allen · Peter Battaglia · Arnaud Doucet · Kimberly Stachenfeld -
2023 : Balanced Training of Energy-Based Models with Adaptive Flow Sampling »
Louis Grenioux · Eric Moulines · Marylou Gabrié -
2023 : Categorical SDEs with Simplex Diffusion »
Pierre Richemond · Sander Dieleman · Arnaud Doucet -
2023 Poster: Conformal Prediction for Federated Uncertainty Quantification Under Label Shift »
Vincent Plassier · Mehdi Makni · Aleksandr Rubashevskii · Eric Moulines · Maxim Panov -
2023 Poster: Fast Rates for Maximum Entropy Exploration »
Daniil Tiapkin · Denis Belomestny · Daniele Calandriello · Eric Moulines · Remi Munos · Alexey Naumov · Pierre Perrault · Yunhao Tang · Michal Valko · Pierre Menard -
2023 Poster: On Sampling with Approximate Transport Maps »
Louis Grenioux · Alain Oliviero Durmus · Eric Moulines · Marylou Gabrié -
2023 Oral: Quantile Credit Assignment »
Thomas Mesnard · Wenqi Chen · Alaa Saade · Yunhao Tang · Mark Rowland · Theophane Weber · Clare Lyle · Audrunas Gruslys · Michal Valko · Will Dabney · Georg Ostrovski · Eric Moulines · Remi Munos -
2023 Poster: Quantile Credit Assignment »
Thomas Mesnard · Wenqi Chen · Alaa Saade · Yunhao Tang · Mark Rowland · Theophane Weber · Clare Lyle · Audrunas Gruslys · Michal Valko · Will Dabney · Georg Ostrovski · Eric Moulines · Remi Munos -
2023 Poster: Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC »
Yilun Du · Conor Durkan · Robin Strudel · Josh Tenenbaum · Sander Dieleman · Rob Fergus · Jascha Sohl-Dickstein · Arnaud Doucet · Will Grathwohl -
2023 Poster: State and parameter learning with PARIS particle Gibbs »
Gabriel Cardoso · Yazid Janati el idrissi · Sylvain Le Corff · Eric Moulines · Jimmy Olsson -
2023 Poster: SE(3) diffusion model with application to protein backbone generation »
Jason Yim · Brian Trippe · Valentin De Bortoli · Emile Mathieu · Arnaud Doucet · Regina Barzilay · Tommi Jaakkola -
2022 Poster: From Dirichlet to Rubin: Optimistic Exploration in RL without Bonuses »
Daniil Tiapkin · Denis Belomestny · Eric Moulines · Alexey Naumov · Sergey Samsonov · Yunhao Tang · Michal Valko · Pierre Menard -
2022 Oral: From Dirichlet to Rubin: Optimistic Exploration in RL without Bonuses »
Daniil Tiapkin · Denis Belomestny · Eric Moulines · Alexey Naumov · Sergey Samsonov · Yunhao Tang · Michal Valko · Pierre Menard -
2022 Poster: Diffusion bridges vector quantized variational autoencoders »
Max Cohen · Guillaume QUISPE · Sylvain Le Corff · Charles Ollion · Eric Moulines -
2022 Spotlight: Diffusion bridges vector quantized variational autoencoders »
Max Cohen · Guillaume QUISPE · Sylvain Le Corff · Charles Ollion · Eric Moulines -
2021 Poster: Differentiable Particle Filtering via Entropy-Regularized Optimal Transport »
Adrien Corenflos · James Thornton · George Deligiannidis · Arnaud Doucet -
2021 Poster: DG-LMC: A Turn-key and Scalable Synchronous Distributed MCMC Algorithm via Langevin Monte Carlo within Gibbs »
Vincent Plassier · Maxime Vono · Alain Durmus · Eric Moulines -
2021 Oral: DG-LMC: A Turn-key and Scalable Synchronous Distributed MCMC Algorithm via Langevin Monte Carlo within Gibbs »
Vincent Plassier · Maxime Vono · Alain Durmus · Eric Moulines -
2021 Oral: Differentiable Particle Filtering via Entropy-Regularized Optimal Transport »
Adrien Corenflos · James Thornton · George Deligiannidis · Arnaud Doucet -
2021 Poster: Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding »
Yangjun Ruan · Karen Ullrich · Daniel Severo · James Townsend · Ashish Khisti · Arnaud Doucet · Alireza Makhzani · Chris Maddison -
2021 Oral: Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding »
Yangjun Ruan · Karen Ullrich · Daniel Severo · James Townsend · Ashish Khisti · Arnaud Doucet · Alireza Makhzani · Chris Maddison -
2021 Poster: Counterfactual Credit Assignment in Model-Free Reinforcement Learning »
Thomas Mesnard · Theophane Weber · Fabio Viola · Shantanu Thakoor · Alaa Saade · Anna Harutyunyan · Will Dabney · Thomas Stepleton · Nicolas Heess · Arthur Guez · Eric Moulines · Marcus Hutter · Lars Buesing · Remi Munos -
2021 Spotlight: Counterfactual Credit Assignment in Model-Free Reinforcement Learning »
Thomas Mesnard · Theophane Weber · Fabio Viola · Shantanu Thakoor · Alaa Saade · Anna Harutyunyan · Will Dabney · Thomas Stepleton · Nicolas Heess · Arthur Guez · Eric Moulines · Marcus Hutter · Lars Buesing · Remi Munos -
2020 Poster: Relaxing Bijectivity Constraints with Continuously Indexed Normalising Flows »
Rob Cornish · Anthony Caterini · George Deligiannidis · Arnaud Doucet -
2020 Poster: Fast and Consistent Learning of Hidden Markov Models by Incorporating Non-Consecutive Correlations »
Robert Mattila · Cristian R. Rojas · Eric Moulines · Vikram Krishnamurthy · Bo Wahlberg -
2019 Poster: Replica Conditional Sequential Monte Carlo »
Alex Shestopaloff · Arnaud Doucet -
2019 Poster: Scalable Metropolis-Hastings for Exact Bayesian Inference with Large Datasets »
Rob Cornish · Paul Vanetti · Alexandre Bouchard-Côté · George Deligiannidis · Arnaud Doucet -
2019 Oral: Replica Conditional Sequential Monte Carlo »
Alex Shestopaloff · Arnaud Doucet -
2019 Oral: Scalable Metropolis-Hastings for Exact Bayesian Inference with Large Datasets »
Rob Cornish · Paul Vanetti · Alexandre Bouchard-Côté · George Deligiannidis · Arnaud Doucet -
2019 Poster: On the Impact of the Activation function on Deep Neural Networks Training »
Soufiane Hayou · Arnaud Doucet · Judith Rousseau -
2019 Oral: On the Impact of the Activation function on Deep Neural Networks Training »
Soufiane Hayou · Arnaud Doucet · Judith Rousseau