Physics-Informed Residual Flows

Physics-Informed Neural Networks (PINNs) embed physical laws into deep learning models. However, conventional PINNs often suffer from failure modes leading to inaccurate solutions. We trace these failure modes to two structural pathologies: gradient shattering, where gradients degrade with depth and provide little training signal, and flow mismatch, where training pushes predictions along trajectories that diverge from the PDE solution path. We introduce ResPINNs, which reformulate PINNs as residual flows, networks that iteratively refine their own predictions through explicit corrective steps, in the spirit of classical iterative solvers. Our analysis shows that this design mitigates both pathologies by keeping updates aligned with descent and by preserving informative gradients across depth. Extensive experiments on PDE benchmarks confirm that ResPINNs achieve higher accuracy with substantially fewer parameters than conventional architectures.

Physics-Informed Neural Networks, or PINNs, use neural networks to solve scientific problems by training them to obey physical laws. However, standard PINNs can fail on difficult equations, producing inaccurate solutions even when the governing physics is known. We find that two issues help explain these failures: useful training signals can fade as they pass through deep networks, and the network’s updates can move predictions away from the path followed by the true physical solution. We address this by adapting ideas from residual networks and neural ordinary differential equations to the PINN setting. Instead of asking one network to produce the solution in a single pass, ResPINNs refine their predictions through a sequence of corrective steps. This makes the model closer in spirit to classical iterative solvers, which start from an initial guess and repeatedly improve it. Our analysis shows that these correction steps preserve stronger training signals and keep the learning process better aligned with the physical solution. Across several benchmark problems, ResPINNs achieve more accurate solutions while using far fewer parameters than standard PINN architectures. This makes them a practical step toward more reliable neural methods for solving scientific and engineering problems governed by physical laws.