On Uniform Error Bounds for Kernel Regression under Non-Gaussian Noise

Providing non-conservative uncertainty quantification for function estimates derived from noisy observations remains a fundamental challenge in statistical machine learning, particularly for applications in safety-critical domains. In this work, we propose novel non-asymptotic probabilistic uniform error bounds for kernel-based regression. Compared to related bounds in the literature that are restricted to (conditionally) independent sub-Gaussian noise, our bounds allow to consider a broad class of non-Gaussian distributions, such as sub-Gaussian, bounded, sub-exponential, and variance/moment-bounded noise. Moreover, our results apply to correlated and uncorrelated noise. We compare our proposed error bounds with existing results in terms of the induced uncertainty region and their performance in safe control, demonstrating the tightness of the proposed bounds.

Learning unknown relationships from measurements is fundamental to many modern technologies, enabling us to predict the behavior of complicated systems and allowing machines to make decisions autonomously. Crucially, in safety‑critical applications like robotics or medical devices, it is essential to not only make accurate predictions, but also to know how uncertain those predictions are. Uncertainty can stem from either limited data coverage (not having seen enough examples of the unknown relationship) or from noise in the measurements. In this paper, we exploit this separation of uncertainty to derive new probabilistic error bounds for predictions made using kernel-based regression (a widely used machine‑learning method). Our focus is on providing reliable and not overly cautious error bounds for several classes of noise distributions common in practice. Through numerical experiments, we show that our approach can produce tighter error bounds than those of existing techniques. This enables a safer and more efficient decision‑making in learning‑based systems.