Fractional is Better: Learnable Derivative Orders in Neural Operator Learning

Neural operators learn mappings between function spaces, enabling fast surrogate solutions to partial differential equations. Despite remarkable architectural diversity, these methods often share a common input representation: raw coordinate-value pairs $(x, u(x))$. We ask whether inputs aligned with PDE differential structure can improve learning. Through Picard iteration on mild solutions, we show that derivatives of the input appear explicitly in the solution operator, suggesting that providing derivative features should reduce the network's implicit differentiation burden. We prove this intuition: providing derivative features improves approximation rates from $O(W^{-(s-m)/d})$ to $O(W^{-s/d})$, where $W$ is network width, $s$ is input regularity, $m$ is the PDE order, and $d$ is spatial dimension. Our central finding, however, is a surprise: the optimal derivative order $\beta^*$ is strictly less than the PDE order $m$. This gap arises from a bias-variance tradeoff in spectral space that we characterize in closed form. Learning $\beta$ from data achieves automatic spectral regularization. We introduce $\partial$-NO (del-NO), for derivative-augmented neural operators, an architecture-agnostic augmentation that provides learnable fractional derivative features to any neural operator backbone. Across benchmark problems and architectures, $\partial$-NO consistently improves prediction accuracy, with learned orders that reflect a representation of known physics modulated by noise and finite-sample constraints.

Physical phenomena around us follow simple rules: the rate of change of one quantity is tied to another, written down as partial differential equations. AI models can learn to predict these phenomena from data, but they only see raw values at points in space. To recover the physics, the model has to figure out the rates of change on its own, which is a hard thing to ask. A natural fix is to compute these rates of change in advance and feed them to the model as extra inputs. The catch is that taking a rate of change also amplifies any noise in the data, so the most physically faithful input is also the noisiest one. We let the model choose its own balance, using a tool called fractional calculus that smoothly tunes how strong each rate-of-change feature is. Across many physics problems and many AI architectures, this small change consistently improves accuracy. The model naturally picks a softer rate of change when data is noisy, and a sharper one when data is clean. The broader lesson is that when bringing science into AI, the textbook answer is rarely the statistically optimal one.