Timezone: »
Amortized variational inference (AVI) replaces instance-specific local inference with a global inference network. While AVI has enabled efficient training of deep generative models such as variational autoencoders (VAE), recent empirical work suggests that inference networks can produce suboptimal variational parameters. We propose a hybrid approach, to use AVI to initialize the variational parameters and run stochastic variational inference (SVI) to refine them. Crucially, the local SVI procedure is itself differentiable, so the inference network and generative model can be trained end-to-end with gradient-based optimization. This semi-amortized approach enables the use of rich generative models without experiencing the posterior-collapse phenomenon common in training VAEs for problems like text generation. Experiments show this approach outperforms strong autoregressive and variational baselines on standard text and image datasets.
Author Information
Yoon Kim (Harvard University)
Sam Wiseman (Harvard University)
Andrew Miller (Harvard)
David Sontag (Massachusetts Institute of Technology)
Alexander Rush (Harvard University)
Related Events (a corresponding poster, oral, or spotlight)
-
2018 Oral: Semi-Amortized Variational Autoencoders »
Fri. Jul 13th 09:20 -- 09:40 AM Room A7
More from the Same Authors
-
2022 : Evaluating Robustness to Dataset Shift via Parametric Robustness Sets »
Michael Oberst · Nikolaj Thams · David Sontag -
2022 : Evaluating Robustness to Dataset Shift via Parametric Robustness Sets »
Nikolaj Thams · Michael Oberst · David Sontag -
2022 Poster: Sample Efficient Learning of Predictors that Complement Humans »
Mohammad-Amin Charusaie · Hussein Mozannar · David Sontag · Samira Samadi -
2022 Poster: Co-training Improves Prompt-based Learning for Large Language Models »
Hunter Lang · Monica Agrawal · Yoon Kim · David Sontag -
2022 Spotlight: Sample Efficient Learning of Predictors that Complement Humans »
Mohammad-Amin Charusaie · Hussein Mozannar · David Sontag · Samira Samadi -
2022 Spotlight: Co-training Improves Prompt-based Learning for Large Language Models »
Hunter Lang · Monica Agrawal · Yoon Kim · David Sontag -
2021 Poster: Neural Pharmacodynamic State Space Modeling »
Zeshan Hussain · Rahul G. Krishnan · David Sontag -
2021 Poster: Regularizing towards Causal Invariance: Linear Models with Proxies »
Michael Oberst · Nikolaj Thams · Jonas Peters · David Sontag -
2021 Poster: Graph Cuts Always Find a Global Optimum for Potts Models (With a Catch) »
Hunter Lang · David Sontag · Aravindan Vijayaraghavan -
2021 Spotlight: Regularizing towards Causal Invariance: Linear Models with Proxies »
Michael Oberst · Nikolaj Thams · Jonas Peters · David Sontag -
2021 Oral: Graph Cuts Always Find a Global Optimum for Potts Models (With a Catch) »
Hunter Lang · David Sontag · Aravindan Vijayaraghavan -
2021 Spotlight: Neural Pharmacodynamic State Space Modeling »
Zeshan Hussain · Rahul G. Krishnan · David Sontag -
2020 Poster: Emergence of Separable Manifolds in Deep Language Representations »
Jonathan Mamou · Hang Le · Miguel A del Rio Fernandez · Cory Stephenson · Hanlin Tang · Yoon Kim · SueYeon Chung -
2020 Poster: Estimation of Bounds on Potential Outcomes For Decision Making »
Maggie Makar · Fredrik Johansson · John Guttag · David Sontag -
2020 Poster: Empirical Study of the Benefits of Overparameterization in Learning Latent Variable Models »
Rares-Darius Buhai · Yoni Halpern · Yoon Kim · Andrej Risteski · David Sontag -
2020 Poster: Consistent Estimators for Learning to Defer to an Expert »
Hussein Mozannar · David Sontag -
2019 Poster: Latent Normalizing Flows for Discrete Sequences »
Zachary Ziegler · Alexander Rush -
2019 Oral: Latent Normalizing Flows for Discrete Sequences »
Zachary Ziegler · Alexander Rush -
2019 Poster: Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models »
Michael Oberst · David Sontag -
2019 Poster: Tensor Variable Elimination for Plated Factor Graphs »
Fritz Obermeyer · Elias Bingham · Martin Jankowiak · Neeraj Pradhan · Justin Chiu · Alexander Rush · Noah Goodman -
2019 Oral: Tensor Variable Elimination for Plated Factor Graphs »
Fritz Obermeyer · Elias Bingham · Martin Jankowiak · Neeraj Pradhan · Justin Chiu · Alexander Rush · Noah Goodman -
2019 Oral: Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models »
Michael Oberst · David Sontag -
2018 Poster: Weightless: Lossy weight encoding for deep neural network compression »
Brandon Reagen · Udit Gupta · Bob Adolf · Michael Mitzenmacher · Alexander Rush · Gu-Yeon Wei · David Brooks -
2018 Poster: Adversarially Regularized Autoencoders »
Jake Zhao · Yoon Kim · Kelly Zhang · Alexander Rush · Yann LeCun -
2018 Oral: Weightless: Lossy weight encoding for deep neural network compression »
Brandon Reagen · Udit Gupta · Bob Adolf · Michael Mitzenmacher · Alexander Rush · Gu-Yeon Wei · David Brooks -
2018 Oral: Adversarially Regularized Autoencoders »
Jake Zhao · Yoon Kim · Kelly Zhang · Alexander Rush · Yann LeCun -
2017 Poster: Image-to-Markup Generation with Coarse-to-Fine Attention »
Yuntian Deng · Anssi Kanervisto · Jeffrey Ling · Alexander Rush -
2017 Poster: Estimating individual treatment effect: generalization bounds and algorithms »
Uri Shalit · Fredrik D Johansson · David Sontag -
2017 Talk: Estimating individual treatment effect: generalization bounds and algorithms »
Uri Shalit · Fredrik D Johansson · David Sontag -
2017 Talk: Image-to-Markup Generation with Coarse-to-Fine Attention »
Yuntian Deng · Anssi Kanervisto · Jeffrey Ling · Alexander Rush -
2017 Poster: Variational Boosting: Iteratively Refining Posterior Approximations »
Andrew Miller · Nicholas J Foti · Ryan P. Adams -
2017 Talk: Variational Boosting: Iteratively Refining Posterior Approximations »
Andrew Miller · Nicholas J Foti · Ryan P. Adams -
2017 Poster: Simultaneous Learning of Trees and Representations for Extreme Classification and Density Estimation »
Yacine Jernite · Anna Choromanska · David Sontag -
2017 Talk: Simultaneous Learning of Trees and Representations for Extreme Classification and Density Estimation »
Yacine Jernite · Anna Choromanska · David Sontag