Fast Mixture of Curvature-Aware Experts for Diverse and Dynamic Graph Topologies

Dynamic graph learning, which focuses on modeling the merging, vanishing, and reconnection of nodes and edges, is crucial for real-world applications. In dynamic graphs, node neighborhoods often exhibit diverse and time-evolving topologies, including hierarchical, grid-like, and cyclic patterns. Existing methods typically embed graphs into a single curvature space, which limits the quality of node representations when the embedding geometry is not aligned well with the local graph topology. In this paper, we propose DyGMoCE, a Dynamic Graph Transformer with a Mixture of Curvature-aware Experts, which efficiently embeds each node at every timestamp into an adaptive curvature space. Specifically, DyGMoCE incorporates a mixture-of-experts framework to both the attention and feed-forward modules, where each expert operates on a Riemannian manifold with a distinct curvature. Then, motivated by the geometric continuity across the experts, we introduce a routing mechanism with a ranking constraint. To improve efficiency, we design a fast Riemannian attention module for DyGMoCE, achieving an average speedup of 27.5% and memory reduction of 52.6%. Notably, the fast Riemannian attention module is broadly applicable to Transformer models with sequence inputs. Extensive experimental results show that DyGMoCE significantly outperforms other state-of-the-art methods.

Many important systems can be represented as graphs: people connected in a social network, flights connected in air traffic, or countries connected through trade. These connections change over time, and the local shape around one item can look very different from another: some neighborhoods are tree-like, some resemble grids, and others form loops. Most existing learning methods describe all of these changing structures using one fixed kind of geometry, which can make them miss important patterns. We propose DyGMoCE, a model that lets each node choose a more suitable geometric view at each point in time. Instead of forcing every neighborhood into the same representation space, the model combines several specialist components, each designed for a different kind or strength of curvature, and routes each node to the most appropriate nearby specialists. We also introduce a faster way to perform the model’s geometric attention calculations, reducing computation and memory use. This makes it easier to learn from large, changing networks whose structure is not uniform. Across eight benchmark datasets, DyGMoCE achieves better performance than strong existing methods, suggesting that adaptive geometry can improve how machine learning systems understand dynamic relationships.