Taming the Loss Landscape of PINNs with Noisy Feynman–Kac Supervision: Operator Preconditioning and Non-Asymptotic Error Bounds

Physics-Informed Neural Networks (PINNs) often train slowly or fail to converge on challenging partial differential equations (PDEs), a behavior recently linked to severely ill-conditioned loss landscapes inherited from the underlying differential operator. We study PINNs augmented with a pointwise data-fidelity term, added at a few points in the domain to the standard residual and boundary losses. We show that this supervision term acts as an operator-level preconditioner: for suitable weights, our comparison bounds guarantee a substantially smaller condition number than under the standard PINN loss, independently of how the pointwise labels are obtained. For a broad class of PDEs admitting a Feynman-Kac (FK) representation, we generate such labels by Monte Carlo averages of the FK functional, resulting in what we call "FK-PINNs", and using the excess risk decomposition approach, we derive non-asymptotic $L^2(\Omega)$-error bounds for FK-PINNs with $\tanh$ activation trained by finitely many steps of gradient descent. Along the way, we establish pseudo-dimension bounds for first- and second-order derivatives of $\tanh$ neural networks, which are of independent interest and, to the best of our knowledge, new. Numerical experiments on Poisson, Schrödinger, mean exit time, and committor problems corroborate the theory, and show that FK-PINNs can successfully solve PDEs for which standard PINNs exhibit severe failure modes.

Partial differential equations (PDEs) model most real-life physical phenomena, and being able to solve them has an immense range of practical applications. Typically, those equations are too complicated to solve by hand, and one resorts to numerical approximation algorithms, which perform well but are not very flexible and become prohibitively costly as the complexity of the PDE increases. Researchers have recently proposed using ideas from deep learning to numerically solve PDEs, which allows to bypass a lot of the limitations the traditional algorithms suffer from. This approach, which consists in training a neural network to predict from "physical data" the solution of a PDE, is commonly referred to as "PINN" (for "physics-informed neural networks"). While promising however, PINNs face a number of challenge which remains to be solved. One of those is the difficulty of "training" the PINN, i.e. getting it to properly "learn" the solution of the PDE. To address these difficulties, we propose in this paper to train PINNs not by just using the "data" from the equation one wants to solve, but also "data" from the actual solution of the equation. We provide an algorithm to approximately compute such data, and we show, both mathematically and through experiments that this indeed has the effect of vastly improving the training process of PINNs, thus making a step towards new efficient algorithms to numerically solve challenging PDEs.