Timezone: »
Variational inference transforms posterior inference into parametric optimization thereby enabling the use of latent variable models where otherwise impractical. However, variational inference can be finicky when different variational parameters control variables that are strongly correlated under the model. Traditional natural gradients based on the variational approximation fail to correct for correlations when the approximation is not the true posterior. To address this, we construct a new natural gradient called the Variational Predictive Natural Gradient (VPNG). Unlike traditional natural gradients for variational inference, this natural gradient accounts for the relationship between model parameters and variational parameters. We demonstrate the insight with a simple example as well as the empirical value on a classification task, a deep generative model of images, and probabilistic matrix factorization for recommendation.
Author Information
Da Tang (Columbia University)
Rajesh Ranganath (New York University)
Related Events (a corresponding poster, oral, or spotlight)
-
2019 Oral: The Variational Predictive Natural Gradient »
Wed Jun 12th 11:35 -- 11:40 PM Room Room 101
More from the Same Authors
-
2019 Poster: Predicate Exchange: Inference with Declarative Knowledge »
Zenna Tavares · Javier Burroni · Edgar Minasyan · Armando Solar-Lezama · Rajesh Ranganath -
2019 Oral: Predicate Exchange: Inference with Declarative Knowledge »
Zenna Tavares · Javier Burroni · Edgar Minasyan · Armando Solar-Lezama · Rajesh Ranganath -
2019 Poster: Correlated Variational Auto-Encoders »
Da Tang · Dawen Liang · Tony Jebara · Nicholas Ruozzi -
2019 Oral: Correlated Variational Auto-Encoders »
Da Tang · Dawen Liang · Tony Jebara · Nicholas Ruozzi -
2018 Poster: Noisin: Unbiased Regularization for Recurrent Neural Networks »
Adji Bousso Dieng · Rajesh Ranganath · Jaan Altosaar · David Blei -
2018 Oral: Noisin: Unbiased Regularization for Recurrent Neural Networks »
Adji Bousso Dieng · Rajesh Ranganath · Jaan Altosaar · David Blei