Timezone: »
Sequential learning with Gaussian processes (GPs) is challenging when access to past data is limited, for example, in continual and active learning. In such cases, errors can accumulate over time due to inaccuracies in the posterior, hyperparameters, and inducing points, making accurate learning challenging. Here, we present a method to keep all such errors in check using the recently proposed dual sparse variational GP. Our method enables accurate inference for generic likelihoods and improves learning by actively building and updating a memory of past data. We demonstrate its effectiveness in several applications involving Bayesian optimization, active learning, and continual learning.
Author Information
Paul Chang (Aalto University)
Prakhar Verma (Aalto University)
ST John (Aalto University, Finnish Center for Artificial Intelligence)
Arno Solin (Aalto University)
Dr. Arno Solin is Assistant Professor in Machine Learning at the Department of Computer Science, Aalto University, Finland, and Adjunct Professor (Docent) at Tampere University, Finland. His research focuses on probabilistic models combining statistical machine learning and signal processing with applications in sensor fusion, robotics, computer vision, and online decision making. He has published around 50 peer-reviewed articles and one book. Previously, he has been a visiting researcher at Uppsala University (2019), University of Cambridge (2017-2018), and University of Sheffield (2014), and worked as a Team Lead in a tech startup. Prof. Solin is the winner of several prizes, hackathons, and modelling competitions, including the Schizophrenia Classification Challenge on Kaggle and the ISIF Jean-Pierre Le Cadre Best Paper Award. Homepage: http://arno.solin.fi
Khan Emtiyaz (RIKEN)
Related Events (a corresponding poster, oral, or spotlight)
-
2023 Oral: Memory-Based Dual Gaussian Processes for Sequential Learning »
Fri. Jul 28th 01:32 -- 01:40 AM Room Meeting Room 316 A-C
More from the Same Authors
-
2023 : Beyond Intuition, a Framework for Applying GPs to Real-World Data »
Kenza Tazi · Jihao Andreas Lin · ST John · Hong Ge · Richard E Turner · Ross Viljoen · Alex Gardner -
2023 : Sparse Function-space Representation of Neural Networks »
Aidan Scannell · Riccardo Mereu · Paul Chang · Ella Tamir · Joni Pajarinen · Arno Solin -
2023 : Memory Maps to Understand Models »
Dharmesh Tailor · Paul Chang · Siddharth Swaroop · Eric Nalisnick · Arno Solin · Khan Emtiyaz -
2023 Workshop: Duality Principles for Modern Machine Learning »
Thomas Moellenhoff · Zelda Mariet · Mathieu Blondel · Khan Emtiyaz -
2023 Poster: Improving Hyperparameter Learning under Approximate Inference in Gaussian Process Models »
Rui Li · ST John · Arno Solin -
2023 Poster: Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning »
Wu Lin · Valentin Duruisseaux · Melvin Leok · Frank Nielsen · Khan Emtiyaz · Mark Schmidt -
2023 Poster: Causal Modeling of Policy Interventions From Treatment–Outcome Sequences »
Çağlar Hızlı · ST John · Anne Juuti · Tuure Saarinen · Kirsi Pietiläinen · Pekka Marttinen -
2021 Poster: Tractable structured natural-gradient descent using local parameterizations »
Wu Lin · Frank Nielsen · Khan Emtiyaz · Mark Schmidt -
2021 Spotlight: Tractable structured natural-gradient descent using local parameterizations »
Wu Lin · Frank Nielsen · Khan Emtiyaz · Mark Schmidt -
2021 Poster: Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning »
Alexander Immer · Matthias Bauer · Vincent Fortuin · Gunnar Ratsch · Khan Emtiyaz -
2021 Spotlight: Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning »
Alexander Immer · Matthias Bauer · Vincent Fortuin · Gunnar Ratsch · Khan Emtiyaz -
2020 : Open discussion in breakout rooms »
Shakir Mohamed · Faris Gezahegn · ST John -
2020 Poster: State Space Expectation Propagation: Efficient Inference Schemes for Temporal Gaussian Processes »
William Wilkinson · Paul Chang · Michael Andersen · Arno Solin -
2020 Poster: Scalable Exact Inference in Multi-Output Gaussian Processes »
Wessel Bruinsma · Eric Perim Martins · William Tebbutt · Scott Hosking · Arno Solin · Richard E Turner -
2020 Social: Queer in AI Social (I) »
Alex Markham · ST John -
2020 : Outro / move to discussion groups »
ST John -
2020 : Intro »
ST John -
2020 Affinity Workshop: Queer in AI »
ST John · William Agnew · Anja Meunier · Alex Markham · Manu Saraswat · Andrew McNamara · Raphael Gontijo Lopes -
2020 Tutorial: Machine Learning with Signal Processing »
Arno Solin -
2019 Poster: End-to-End Probabilistic Inference for Nonstationary Audio Analysis »
William Wilkinson · Michael Riis Andersen · Joshua D. Reiss · Dan Stowell · Arno Solin -
2019 Oral: End-to-End Probabilistic Inference for Nonstationary Audio Analysis »
William Wilkinson · Michael Riis Andersen · Joshua D. Reiss · Dan Stowell · Arno Solin -
2018 Poster: Large-Scale Cox Process Inference using Variational Fourier Features »
ST John · James Hensman -
2018 Poster: State Space Gaussian Processes with Non-Gaussian Likelihood »
Hannes Nickisch · Arno Solin · Alexander Grigorevskiy -
2018 Oral: Large-Scale Cox Process Inference using Variational Fourier Features »
ST John · James Hensman -
2018 Oral: State Space Gaussian Processes with Non-Gaussian Likelihood »
Hannes Nickisch · Arno Solin · Alexander Grigorevskiy