Learning Stochastic Dynamical Systems as an Implicit Regularization with Graph Neural Network

We provide a stochastic Gumbel Graph Network (S-GGN) for the long-term prediction of high-dimensional time series data with complex noise and spatial correlations. Specifically, by training the drift and diffusion terms of a stochastic differential equation (SDE) with a Gumbel Matrix embedding, we can capture both the randomness and underlying spatial dynamics of input data. This framework allows us to investigate the implicit regularization effect of noise in S-GGN. First, we derive the difference between the two corresponding loss functions in a small neighborhood of weight. Then, we employ Kuramoto's model to generate data for comparing the spectral density from the Hessian Matrix of the two loss functions. Moreover, we also apply our model to real-world scenarios such as wireless communication data. Experiment results demonstrate that S-GGN exhibits better convergence, robustness, and generalization.