Smoothness Errors in Dynamics Models and How to Avoid Them

Modern neural networks have shown promise for solving partial differential equations over surfaces, often by discretizing the surface as a mesh and learning with a mesh-aware graph neural network. However, graph neural networks suffer from oversmoothing, where a node's features become increasingly similar to those of its neighbors. Unitary graph convolutions, which are mathematically constrained to preserve smoothness, have been proposed to address this issue. Despite this, in many physical systems, such as diffusion processes, smoothness naturally increases and unitarity may be overconstraining. In this paper, we systematically study the smoothing effects of different GNNs for dynamics modeling and prove that unitary convolutions hurt performance for such tasks. We propose relaxed unitary convolutions that balance smoothness preservation with the natural smoothing required for physical systems. We also generalize unitary and relaxed unitary convolutions from graphs to meshes. In experiments on PDEs such as the heat and wave equations over complex meshes and on weather forecasting, we find that our method outperforms several strong baselines, including mesh-aware transformers and equivariant neural networks. Our code is available at github.com/EdwardBerman/rayleigh_analysis

We propose approximately unitary graph convolutions, a neural network layer and architecture for modeling dynamical systems over meshes with inductive biases that are designed to match the underlying smoothness of the system. First, we motivate our analysis be deriving an approximation error lower bound on the approximation error of smoothness preserving unitary functions, demonstrating that they are overly restrictive when predicting dynamics with high angular dependence in the solution's norm. We then introduce relaxed unitary convolutions that balance accuracy with smoothness preservation, and extend both existing smoothness metrics and the unitary convolution framework to meshes. Finally, we empirically analyze the smoothness behavior of various GNN architectures on complex dynamical systems, showing that controlling smoothness can match or outperform strong baselines.