Robust Bayesian Optimisation with Unbounded Corruptions

Bayesian Optimization is critically vulnerable to extreme outliers. Existing provably robust methods typically assume a bounded cumulative corruption budget, which makes them defenseless against even a single corruption of sufficient magnitude. To address this, we introduce a new adversary whose budget is only bounded in the frequency of corruptions, not in their magnitude. We then derive RCGP-UCB, an algorithm coupling the upper confidence bound (UCB) approach with a Robust Conjugate Gaussian Process (RCGP). We present stable and adaptive versions of RCGP-UCB, and prove that they achieve sublinear regret in the presence of up to $O(T^{1/4})$ and $O(T^{1/7})$ corruptions with possibly infinite magnitude. This robustness comes at near zero cost: without outliers, RCGP-UCB's regret bounds match those of the standard GP-UCB algorithm.

Bayesian Optimization is a powerful mathematical tool used to find optimal solutions in complex scenarios like drug discovery or AI tuning. However, standard methods are critically vulnerable to extreme data outliers, like sensor glitches or experimental failures. Previous fixes assumed the total size of all errors had a strict known limit, leaving them defenseless against a single massive anomaly. Our paper adapts robust models from the Generalised-Bayes literature, which provide theoretical guarantees against extreme corruptions. By leveraging these models, we propose algorithms capable of surviving corruptions of infinite magnitude, provided they are limited in frequency. To ensure this robustness doesn't come at the cost of performance in uncorrupted environments, we introduce a "Trust Plateau." This mechanism trusts data falling within an expected range, which maintains standard efficiency, while mathematically down-weighting and ignoring extreme anomalies.