ICML 2020

Timezone: »

Poster

Scalable Exact Inference in Multi-Output Gaussian Processes

Wessel Bruinsma · Eric Perim Martins · William Tebbutt · Scott Hosking · Arno Solin · Richard E Turner

Wed Jul 15 12:00 PM -- 12:45 PM & Thu Jul 16 01:00 AM -- 01:45 AM (PDT) @ Virtual

in Poster Session 26 »

Multi-output Gaussian processes (MOGPs) leverage the flexibility and interpretability of GPs while capturing structure across outputs, which is desirable, for example, in spatio-temporal modelling. The key problem with MOGPs is their computational scaling $O(n^3 p^3)$, which is cubic in the number of both inputs $n$ (e.g., time points or locations) and outputs $p$. For this reason, a popular class of MOGPs assumes that the data live around a low-dimensional linear subspace, reducing the complexity to $O(n^3 m^3)$. However, this cost is still cubic in the dimensionality of the subspace $m$, which is still prohibitively expensive for many applications. We propose the use of a sufficient statistic of the data to accelerate inference and learning in MOGPs with orthogonal bases. The method achieves linear scaling in $m$ in practice, allowing these models to scale to large $m$ without sacrificing significant expressivity or requiring approximation. This advance opens up a wide range of real-world tasks and can be combined with existing GP approximations in a plug-and-play way. We demonstrate the efficacy of the method on various synthetic and real-world data sets.

Keywords: Gaussian Processes

[ Slides]

Author Information

Wessel Bruinsma (University of Cambridge and Invenia Labs)

Eric Perim Martins (Invenia Labs)

William Tebbutt (University of Cambridge)

Scott Hosking (British Antarctic Survey)

Arno Solin (Aalto University)

Dr. Arno Solin is Assistant Professor in Machine Learning at the Department of Computer Science, Aalto University, Finland, and Adjunct Professor (Docent) at Tampere University, Finland. His research focuses on probabilistic models combining statistical machine learning and signal processing with applications in sensor fusion, robotics, computer vision, and online decision making. He has published around 50 peer-reviewed articles and one book. Previously, he has been a visiting researcher at Uppsala University (2019), University of Cambridge (2017-2018), and University of Sheffield (2014), and worked as a Team Lead in a tech startup. Prof. Solin is the winner of several prizes, hackathons, and modelling competitions, including the Schizophrenia Classification Challenge on Kaggle and the ISIF Jean-Pierre Le Cadre Best Paper Award. Homepage: http://arno.solin.fi

Richard E Turner (University of Cambridge)

Richard Turner holds a Lectureship (equivalent to US Assistant Professor) in Computer Vision and Machine Learning in the Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, UK. He is a Fellow of Christ's College Cambridge. Previously, he held an EPSRC Postdoctoral research fellowship which he spent at both the University of Cambridge and the Laboratory for Computational Vision, NYU, USA. He has a PhD degree in Computational Neuroscience and Machine Learning from the Gatsby Computational Neuroscience Unit, UCL, UK and a M.Sci. degree in Natural Sciences (specialism Physics) from the University of Cambridge, UK. His research interests include machine learning, signal processing and developing probabilistic models of perception.