Goal-Oriented Lower-Tail Calibration of Gaussian Processes for Bayesian Optimization

Bayesian optimization (BO) selects evaluation points for expensive black-box objectives using Gaussian process (GP) predictive distributions. Kernel choice and hyperparameter selection can lead to miscalibrated predictive distributions and an inappropriate exploration--exploitation trade-off. For minimization, sampling criteria such as expected improvement (EI) depend on the predictive distribution below the current best value, so lower-tail miscalibration directly affects the sampling decision. This article studies goal-oriented calibration of GP predictive distributions below a low threshold $t$ in the noiseless setting, for standard GP models with hyperparameters selected by maximum likelihood. A framework for predictive reliability below $t$ is introduced, based on two notions of spatial calibration: occurrence calibration over the design space and thresholded $\mu$-calibration on sublevel sets of the form $\lbrace x\in\mathbb{X}, f(x)\le t \rbrace$. Building on this framework, we propose tcGP, a post-hoc method that calibrates GP predictive distributions below $t$, and we show that the resulting EI-based global optimization algorithm remains dense in the design space. Experiments on standard benchmarks show improved lower-tail calibration and BO performance relative to standard GP models and globally calibrated GP models.

Many scientific and engineering tasks require finding the best setting of a system, such as the shape of a component, the parameters of a simulation, or the settings of a machine learning model. Testing one setting can be slow or expensive, so Bayesian optimization tries to choose each new test carefully. To do this, it builds a model that predicts both the likely result of a test and how uncertain that prediction is. In practice, these uncertainty estimates can be unreliable, especially in the most promising regions where low objective values are expected. We study how to make these predictions more reliable specifically in those promising low-value regions. We introduce a way to measure this targeted reliability and propose a calibration method called tcGP that adjusts Gaussian process predictions with this goal in mind. Experiments on standard optimization problems show that this targeted approach improves both prediction reliability and optimization performance. By helping Bayesian optimization focus its tests more effectively, this method can reduce the number of expensive evaluations needed in practice.