Probabilistic Numerics — Computation is Machine Learning

Machine learning is the process of estimating latent representations or variables from finite data. If the data is insufficient, this inference process leaves a finite estimation error. Probabilistic (Bayesian) machine learning attempts to capture this empirical uncertainty in a probability distribution.

But what actually happens inside of a Learning Machine, the computational side of ML, is invariably the solution of a numerical problem: Optimisation for deep learning, solving differential equations for diffusion, flow matching, and scientific simulation, or even just (large-scale, approximate) numerical linear algebra. These numerical tasks have no analytic solution in reach. The computational resources are insufficient, and so the computation leaves a finite computational error. Probabilistic numerical methods attempt to capture this computational uncertainty in a probability distribution.

By matching the mathematical modelling language of the empirical and the computational side of machine learning in this way, probabilistic numerical methods open new opportunities for computational savings, and new functionality in the ML stack: Computational and data uncertainty can be controlled in relation to each other, and information from data can flow "backwards" through a computation to solve inverse problems. A growing research community within ML is developing this toolchain, typically by building on established, highly efficient, classic numerical methods.

The tutorial is split in three parts. We will start with a simple worked example to establish key concepts and patterns. A second part will generalise these insights into a design pattern across a large class of numerical tasks. Finally, a hands-on code demo will demonstrate how probabilistic numerical methods work in practice.