Timezone: »

Improving Hyperparameter Learning under Approximate Inference in Gaussian Process Models
Rui Li · ST John · Arno Solin

Thu Jul 27 04:30 PM -- 06:00 PM (PDT) @ Exhibit Hall 1 #212

Approximate inference in Gaussian process (GP) models with non-conjugate likelihoods gets entangled with the learning of the model hyperparameters. We improve hyperparameter learning in GP models and focus on the interplay between variational inference (VI) and the learning target. While VI's lower bound to the marginal likelihood is a suitable objective for inferring the approximate posterior, we show that a direct approximation of the marginal likelihood as in Expectation Propagation (EP) is a better learning objective for hyperparameter optimization. We design a hybrid training procedure to bring the best of both worlds: it leverages conjugate-computation VI for inference and uses an EP-like marginal likelihood approximation for hyperparameter learning. We compare VI, EP, Laplace approximation, and our proposed training procedure and empirically demonstrate the effectiveness of our proposal across a wide range of data sets.

Author Information

Rui Li (Aalto University)
ST John (Aalto University, Finnish Center for Artificial Intelligence)
Arno Solin (Aalto University)
Arno Solin

Dr. Arno Solin is Assistant Professor in Machine Learning at the Department of Computer Science, Aalto University, Finland, and Adjunct Professor (Docent) at Tampere University, Finland. His research focuses on probabilistic models combining statistical machine learning and signal processing with applications in sensor fusion, robotics, computer vision, and online decision making. He has published around 50 peer-reviewed articles and one book. Previously, he has been a visiting researcher at Uppsala University (2019), University of Cambridge (2017-2018), and University of Sheffield (2014), and worked as a Team Lead in a tech startup. Prof. Solin is the winner of several prizes, hackathons, and modelling competitions, including the Schizophrenia Classification Challenge on Kaggle and the ISIF Jean-Pierre Le Cadre Best Paper Award. Homepage: http://arno.solin.fi

More from the Same Authors