Deep Coupling Learning for Solving PDEs

Physics-Informed Neural Networks (PINNs) represent a significant advancement in computational methods for solving partial differential equations (PDEs). However, the adoption of deeper neural network architectures presents significant challenges, as they struggle to address differential-related complications that arise during the computation of derivatives over the input of PINNs. These complications extend beyond traditional vanishing and exploding gradients to include vanishing and exploding differentials, with both phenomena becoming more severe as networks grow deeper. By examining the computation graph of derivatives in deep neural networks, we identify key bottlenecks causing numerical instabilities in deep architectures. In response, we introduce a novel approach that utilizes Coupling Layers with carefully regulated spectral norms of Jacobian matrices to stabilize and facilitate deep PINN training, effectively addressing differential-related challenges and improving model stability. Our proposed architecture successfully mitigates the fundamental constraints of deeper PINNs while maximizing their capabilities through consistent differential propagation. Comprehensive evaluations show that our approach surpasses conventional shallow PINN methods and alternative deep PINN designs across a range of challenging problems.

Many important problems in science and engineering, such as fluid flow, heat transfer, and quantum mechanics, are described by partial differential equations. Physics-informed neural networks try to solve these equations by training a neural network to obey the underlying physical laws, but they often struggle when the true solution contains fine details or rapidly changing patterns. A natural idea is to use deeper neural networks, which are usually more expressive, but in this setting deeper networks can make the required derivative computations unstable. In this paper, we study why this instability happens and propose a new neural network architecture called CoupledNet. CoupledNet is designed to keep derivative information stable as it passes through many layers, allowing deeper physics-informed models to be trained more reliably. Across several challenging equation-solving tasks, especially those with high-frequency or high-dimensional behavior, CoupledNet achieves more accurate and stable results than standard shallow or residual-network-based approaches. This work helps make neural-network-based scientific computing more reliable for complex physical systems.