Timezone: »

 
Poster
Preconditioning for Scalable Gaussian Process Hyperparameter Optimization
Jonathan Wenger · Geoff Pleiss · Philipp Hennig · John Cunningham · Jacob Gardner

Tue Jul 19 03:30 PM -- 05:30 PM (PDT) @ Hall E #737
in Poster Session 1 »

Gaussian process hyperparameter optimization requires linear solves with, and log-determinants of, large kernel matrices. Iterative numerical techniques are becoming popular to scale to larger datasets, relying on the conjugate gradient method (CG) for the linear solves and stochastic trace estimation for the log-determinant. This work introduces new algorithmic and theoretical insights for preconditioning these computations. While preconditioning is well understood in the context of CG, we demonstrate that it can also accelerate convergence and reduce variance of the estimates for the log-determinant and its derivative. We prove general probabilistic error bounds for the preconditioned computation of the log-determinant, log-marginal likelihood and its derivatives. Additionally, we derive specific rates for a range of kernel-preconditioner combinations, showing that up to exponential convergence can be achieved. Our theoretical results enable provably efficient optimization of kernel hyperparameters, which we validate empirically on large-scale benchmark problems. There our approach accelerates training by up to an order of magnitude.

Keywords: PM: Gaussian Processes · MISC: Scalable Algorithms

Author Information

Jonathan Wenger (University of Tübingen)
Geoff Pleiss (Columbia University)
Philipp Hennig (University of Tuebingen)
John Cunningham (Columbia)
Jacob Gardner (University of Pennsylvania)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors