Discriminative Mixture-of-Experts on Graphs with Reliable Expert Fusion

Graph Mixture-of-Experts (Graph-MoE) offers a way to scale GNNs via adaptive capacity allocation, with the goal of allowing different experts to capture diverse graph patterns. Its effectiveness heavily depends on the coordination between routing decisions and expert specialization. However, through extensive empirical study, we identify two critical phenomena. First, discrimination loss occurs on both the expert and routing sides, where GNN experts become highly homogenized and the router collapses to a small subset of experts, failing to reflect diverse graph semantics. Second, routing uncertainty is prevalent, as existing routers produce uncertain expert assignments for most nodes, and such uncertainty exhibits a strong negative correlation with model performance. To address these issues, we propose C$^2$GMoE, a novel Graph-MoE framework featuring Contrastive routing and Confidence-aware fusion. We introduce a group-wise contrastive routing strategy that provides explicit guidance for routing optimization by aligning node-level routing decisions with semantic clusters while satisfying load-balancing constraints. Moreover, through a theoretical analysis of generalization error, we develop a confidence-aware fusion mechanism that adaptively reweights expert predictions according to their confidence. Extensive experiments across multiple benchmarks demonstrate the effectiveness of our proposed C$^2$GMoE.

Graph Neural Networks (GNNs) are powerful AI tools used to analyze interconnected data, such as social networks or molecular structures. To boost their processing power, researchers use a "mixture-of-experts" approach (Graph-MoE), where a central router directs each individual node within the graph network to specialized GNN components ("experts"). However, current systems face two critical bottlenecks: the GNN experts often become identical, losing their ability to capture diverse graph patterns, while the router frequently makes ambiguous node assignments that degrade overall accuracy. To address these issues, we introduce a new framework called C$^2$GMoE. First, it employs a contrastive routing strategy that aligns how nodes are dispatched with their actual positions and meanings in the graph, guiding different experts to genuinely specialize in distinct network structures. Second, to counter dispatcher confusion, our system dynamically evaluates how confident each expert is about every single node and prioritizes the most reliable predictions. Extensive testing demonstrates that our method successfully restores expert specialization, significantly improving how AI learns from complex, networked graph data.