Timezone: »
Approximate inference in Gaussian process (GP) models with non-conjugate likelihoods gets entangled with the learning of the model hyperparameters. We improve hyperparameter learning in GP models and focus on the interplay between variational inference (VI) and the learning target. While VI's lower bound to the marginal likelihood is a suitable objective for inferring the approximate posterior, we show that a direct approximation of the marginal likelihood as in Expectation Propagation (EP) is a better learning objective for hyperparameter optimization. We design a hybrid training procedure to bring the best of both worlds: it leverages conjugate-computation VI for inference and uses an EP-like marginal likelihood approximation for hyperparameter learning. We compare VI, EP, Laplace approximation, and our proposed training procedure and empirically demonstrate the effectiveness of our proposal across a wide range of data sets.
Author Information
Rui Li (Aalto University)
ST John (Aalto University, Finnish Center for Artificial Intelligence)
Arno Solin (Aalto University)
Dr. Arno Solin is Assistant Professor in Machine Learning at the Department of Computer Science, Aalto University, Finland, and Adjunct Professor (Docent) at Tampere University, Finland. His research focuses on probabilistic models combining statistical machine learning and signal processing with applications in sensor fusion, robotics, computer vision, and online decision making. He has published around 50 peer-reviewed articles and one book. Previously, he has been a visiting researcher at Uppsala University (2019), University of Cambridge (2017-2018), and University of Sheffield (2014), and worked as a Team Lead in a tech startup. Prof. Solin is the winner of several prizes, hackathons, and modelling competitions, including the Schizophrenia Classification Challenge on Kaggle and the ISIF Jean-Pierre Le Cadre Best Paper Award. Homepage: http://arno.solin.fi
More from the Same Authors
-
2023 : Beyond Intuition, a Framework for Applying GPs to Real-World Data »
Kenza Tazi · Jihao Andreas Lin · ST John · Hong Ge · Richard E Turner · Ross Viljoen · Alex Gardner -
2023 : Sparse Function-space Representation of Neural Networks »
Aidan Scannell · Riccardo Mereu · Paul Chang · Ella Tamir · Joni Pajarinen · Arno Solin -
2023 : Memory Maps to Understand Models »
Dharmesh Tailor · Paul Chang · Siddharth Swaroop · Eric Nalisnick · Arno Solin · Khan Emtiyaz -
2023 Oral: Memory-Based Dual Gaussian Processes for Sequential Learning »
Paul Chang · Prakhar Verma · ST John · Arno Solin · Khan Emtiyaz -
2023 Poster: Causal Modeling of Policy Interventions From Treatment–Outcome Sequences »
Çağlar Hızlı · ST John · Anne Juuti · Tuure Saarinen · Kirsi Pietiläinen · Pekka Marttinen -
2023 Poster: Memory-Based Dual Gaussian Processes for Sequential Learning »
Paul Chang · Prakhar Verma · ST John · Arno Solin · Khan Emtiyaz -
2020 : Open discussion in breakout rooms »
Shakir Mohamed · Faris Gezahegn · ST John -
2020 Poster: State Space Expectation Propagation: Efficient Inference Schemes for Temporal Gaussian Processes »
William Wilkinson · Paul Chang · Michael Andersen · Arno Solin -
2020 Poster: Scalable Exact Inference in Multi-Output Gaussian Processes »
Wessel Bruinsma · Eric Perim Martins · William Tebbutt · Scott Hosking · Arno Solin · Richard E Turner -
2020 Social: Queer in AI Social (I) »
Alex Markham · ST John -
2020 : Outro / move to discussion groups »
ST John -
2020 : Intro »
ST John -
2020 Affinity Workshop: Queer in AI »
ST John · William Agnew · Anja Meunier · Alex Markham · Manu Saraswat · Andrew McNamara · Raphael Gontijo Lopes -
2020 Tutorial: Machine Learning with Signal Processing »
Arno Solin -
2019 Poster: End-to-End Probabilistic Inference for Nonstationary Audio Analysis »
William Wilkinson · Michael Riis Andersen · Joshua D. Reiss · Dan Stowell · Arno Solin -
2019 Oral: End-to-End Probabilistic Inference for Nonstationary Audio Analysis »
William Wilkinson · Michael Riis Andersen · Joshua D. Reiss · Dan Stowell · Arno Solin -
2018 Poster: Large-Scale Cox Process Inference using Variational Fourier Features »
ST John · James Hensman -
2018 Poster: State Space Gaussian Processes with Non-Gaussian Likelihood »
Hannes Nickisch · Arno Solin · Alexander Grigorevskiy -
2018 Oral: Large-Scale Cox Process Inference using Variational Fourier Features »
ST John · James Hensman -
2018 Oral: State Space Gaussian Processes with Non-Gaussian Likelihood »
Hannes Nickisch · Arno Solin · Alexander Grigorevskiy