Deep Gaussian processes (DGPs) can model complex marginal densities as well as complex mappings. Non-Gaussian marginals are essential for modelling real-world data, and can be generated from the DGP by incorporating uncorrelated variables to the model. Previous work in the DGP model has introduced noise additively, and used variational inference with a combination of sparse Gaussian processes and mean-field Gaussians for the approximate posterior. Additive noise attenuates the signal, and the Gaussian form of variational distribution may lead to an inaccurate posterior. We instead incorporate noisy variables as latent covariates, and propose a novel importance-weighted objective, which leverages analytic results and provides a mechanism to trade off computation for improved accuracy. Our results demonstrate that the importance-weighted objective works well in practice and consistently outperforms classical variational inference, especially for deeper models.
Author Information
Hugh Salimbeni (Imperial College)
Vincent Dutordoir (PROWLER.io)
James Hensman (PROWLER.io)
Marc P Deisenroth (Imperial College London and PROWLER.io)
Related Events (a corresponding poster, oral, or spotlight)
-
2019 Oral: Deep Gaussian Processes with Importance-Weighted Variational Inference »
Wed Jun 12th 12:05 -- 12:10 PM Room Room 101
More from the Same Authors
-
2019 Poster: Overcoming Mean-Field Approximations in Recurrent Gaussian Process Models »
Alessandro Davide Ialongo · Mark van der Wilk · James Hensman · Carl E Rasmussen -
2019 Oral: Overcoming Mean-Field Approximations in Recurrent Gaussian Process Models »
Alessandro Davide Ialongo · Mark van der Wilk · James Hensman · Carl E Rasmussen -
2018 Poster: Large-Scale Cox Process Inference using Variational Fourier Features »
ST John · James Hensman -
2018 Oral: Large-Scale Cox Process Inference using Variational Fourier Features »
ST John · James Hensman -
2018 Poster: Design of Experiments for Model Discrimination Hybridising Analytical and Data-Driven Approaches »
Simon Olofsson · Marc P Deisenroth · Ruth Misener -
2018 Oral: Design of Experiments for Model Discrimination Hybridising Analytical and Data-Driven Approaches »
Simon Olofsson · Marc P Deisenroth · Ruth Misener
