Approximate natural gradient in Gaussian processes with non-log-concave likelihoods

Approximate Bayesian inference on Gaussian process models with non-log-concave likelihoods is a challenging problem. When the log-likelihood function lacks concavity, finding the maximum a posterior estimate of the Gaussian process posterior becomes troublesome. Additionally, the lack of concavity complicates computer implementations and may increase computational load. In this work, we propose using an approximate Fisher information matrix as an alternative for defining a variant of the natural gradient update in the context of Gaussian process modeling, achieving this without incurring additional costs and with less analytical derivations. Moreover, experiments show that the approximate natural gradient works efficiently when the log-likelihood function strongly lacks concavity.