Mitigating Gradient Pathology in PINNs through Aligned Constraint

While Physics-Informed Neural Networks (PINNs) are powerful for solving Partial Differential Equations (PDEs), their training is often paralyzed by gradient pathology. The gradients from the PDE residuals and boundary constraints oppose each other, trapping the model in local minima. Current solutions, such as adaptive weighting or hard constraints, either fail to fundamentally resolve this ill-conditioning or are limited to simple geometries. In this study, we systematically analyze the possible causes of this gradient pathology from the perspectives of loss landscapes and optimization dynamics. Based on the obtained conclusion, we propose Constraint-Aligned loss with Manifold Lifting (CAML). By reformulating all zeroth-order terms into aligned constraints, our method effectively mitigates gradient conflicts. In addition, we introduce a delay factor to help the optimizer skip the high-curvature area. Experiments demonstrate that our CAML significantly enhances numerical stability and efficiency in highly complex PINN problems. Our code is open-sourced on CAML.

Physics-Informed Neural Networks (PINNs) are AI models that use the laws of physics to simulate real-world phenomena without needing massive amounts of training data. However, successfully training these models is notoriously difficult. Often, the model gets stuck because the general rules of physics and the specific conditions of the environment pull the learning process in completely opposite directions. In this paper, we discovered that this conflict happens because the physics rules create a deep "valley" of almost-correct answers that trap the AI in a bad spot. To fix this, we created a new training method called CAML. Instead of forcing the model to solve everything rigidly at once, CAML introduces a flexible mathematical buffer that aligns the physics rules with the environmental conditions. We also delay the introduction of the physics rules at the very beginning of training so the AI gets off to a better start. By stopping this internal tug-of-war, our approach makes training these AI models significantly faster and more reliable, especially for complex real-world simulations.