Timezone: »
Vector Quantized-Variational AutoEncoders (VQ-VAE) are generative models based on discrete latent representations of the data, where inputs are mapped to a finite set of learned embeddings.To generate new samples, an autoregressive prior distribution over the discrete states must be trained separately. This prior is generally very complex and leads to slow generation. In this work, we propose a new model to train the prior and the encoder/decoder networks simultaneously. We build a diffusion bridge between a continuous coded vector and a non-informative prior distribution. The latent discrete states are then given as random functions of these continuous vectors. We show that our model is competitive with the autoregressive prior on the mini-Imagenet and CIFAR dataset and is efficient in both optimization and sampling. Our framework also extends the standard VQ-VAE and enables end-to-end training.
Author Information
Max Cohen (Télécom SudParis, Accenta)
Guillaume QUISPE (Polytechnique)
Sylvain Le Corff (Télécom SudParis)
Charles Ollion (Polytechnique, IPP)
Eric Moulines (Ecole Polytechnique)
Related Events (a corresponding poster, oral, or spotlight)
-
2022 Poster: Diffusion bridges vector quantized variational autoencoders »
Tue. Jul 19th through Wed the 20th Room Hall E #427
More from the Same Authors
-
2021 : Model-Free Approach to Evaluate Reinforcement Learning Algorithms »
Denis Belomestny · Ilya Levin · Eric Moulines · Alexey Naumov · Sergey Samsonov · Veronika Zorina -
2023 : Balanced Training of Energy-Based Models with Adaptive Flow Sampling »
Louis Grenioux · Eric Moulines · Marylou Gabrié -
2023 Poster: Conformal Prediction for Federated Uncertainty Quantification Under Label Shift »
Vincent Plassier · Mehdi Makni · Aleksandr Rubashevskii · Eric Moulines · Maxim Panov -
2023 Poster: Fast Rates for Maximum Entropy Exploration »
Daniil Tiapkin · Denis Belomestny · Daniele Calandriello · Eric Moulines · Remi Munos · Alexey Naumov · Pierre Perrault · Yunhao Tang · Michal Valko · Pierre Menard -
2023 Poster: On Sampling with Approximate Transport Maps »
Louis Grenioux · Alain Oliviero Durmus · Eric Moulines · Marylou Gabrié -
2023 Oral: Quantile Credit Assignment »
Thomas Mesnard · Wenqi Chen · Alaa Saade · Yunhao Tang · Mark Rowland · Theophane Weber · Clare Lyle · Audrunas Gruslys · Michal Valko · Will Dabney · Georg Ostrovski · Eric Moulines · Remi Munos -
2023 Poster: Quantile Credit Assignment »
Thomas Mesnard · Wenqi Chen · Alaa Saade · Yunhao Tang · Mark Rowland · Theophane Weber · Clare Lyle · Audrunas Gruslys · Michal Valko · Will Dabney · Georg Ostrovski · Eric Moulines · Remi Munos -
2023 Poster: State and parameter learning with PARIS particle Gibbs »
Gabriel Cardoso · Yazid Janati el idrissi · Sylvain Le Corff · Eric Moulines · Jimmy Olsson -
2022 Poster: From Dirichlet to Rubin: Optimistic Exploration in RL without Bonuses »
Daniil Tiapkin · Denis Belomestny · Eric Moulines · Alexey Naumov · Sergey Samsonov · Yunhao Tang · Michal Valko · Pierre Menard -
2022 Oral: From Dirichlet to Rubin: Optimistic Exploration in RL without Bonuses »
Daniil Tiapkin · Denis Belomestny · Eric Moulines · Alexey Naumov · Sergey Samsonov · Yunhao Tang · Michal Valko · Pierre Menard -
2021 Poster: Monte Carlo Variational Auto-Encoders »
Achille Thin · Nikita Kotelevskii · Arnaud Doucet · Alain Durmus · Eric Moulines · Maxim Panov -
2021 Spotlight: Monte Carlo Variational Auto-Encoders »
Achille Thin · Nikita Kotelevskii · Arnaud Doucet · Alain Durmus · Eric Moulines · Maxim Panov -
2021 Poster: DG-LMC: A Turn-key and Scalable Synchronous Distributed MCMC Algorithm via Langevin Monte Carlo within Gibbs »
Vincent Plassier · Maxime Vono · Alain Durmus · Eric Moulines -
2021 Oral: DG-LMC: A Turn-key and Scalable Synchronous Distributed MCMC Algorithm via Langevin Monte Carlo within Gibbs »
Vincent Plassier · Maxime Vono · Alain Durmus · Eric Moulines -
2021 Poster: Counterfactual Credit Assignment in Model-Free Reinforcement Learning »
Thomas Mesnard · Theophane Weber · Fabio Viola · Shantanu Thakoor · Alaa Saade · Anna Harutyunyan · Will Dabney · Thomas Stepleton · Nicolas Heess · Arthur Guez · Eric Moulines · Marcus Hutter · Lars Buesing · Remi Munos -
2021 Spotlight: Counterfactual Credit Assignment in Model-Free Reinforcement Learning »
Thomas Mesnard · Theophane Weber · Fabio Viola · Shantanu Thakoor · Alaa Saade · Anna Harutyunyan · Will Dabney · Thomas Stepleton · Nicolas Heess · Arthur Guez · Eric Moulines · Marcus Hutter · Lars Buesing · Remi Munos -
2020 Poster: Fast and Consistent Learning of Hidden Markov Models by Incorporating Non-Consecutive Correlations »
Robert Mattila · Cristian R. Rojas · Eric Moulines · Vikram Krishnamurthy · Bo Wahlberg