GeoEvo: Identity-Aware Potential Game with Geometric Evolution for Personalized Multimodal Federated Learning

We reconceptualize Personalized Multimodal Federated Learning (PMFL) by treating missing modalities as intrinsic structural identities that constrain each client to a distinct Riemannian submanifold, rather than as deficiencies to be compensated. To reconcile the tension between identity preservation and cross-client collaboration, we cast PMFL as an identity-aware potential game and seek a geometry-consistent equilibrium instead of a monolithic full-modality optimum. We propose GeoEvo, a federated approximate solver that combines curvature-adaptive Fisher descent with manifold-lifted evolutionary search: Natural Evolution Strategies for basin escape, and Particle Swarm updates anchored to a server-broadcast Fr\'echet prototype for cross-client transfer. A monotone acceptance rule drives per-step potential dissipation, yielding an $\mathcal{O}(1/\sqrt{T})$ stationarity rate and convergence toward first-order Nash equilibria in non-convex regimes. Empirically, GeoEvo improves personalization and robustness across diverse modality-missing patterns.

In collaborative networks (e.g., hospitals jointly predicting behavior), participants collect divergent data—some images, some text, few both—pushing each model onto a distinct curved surface, rendering a one-size-fits-all model unworkable. Existing approaches treat these missing modalities as flaws, while we see them as fundamental to each participant’s identity; forcing a uniform model ignores hidden geometric limits, creating severe optimization conflicts and degraded local performance. To bridge this gap without erasing individual differences, we model the problem as a potential game: all clients pursue their own goals while implicitly descending a shared landscape, guaranteeing stable coordination. Our method, GeoEvo, uses a central server to share lightweight geometric checkpoints across these distinct surfaces, while each client runs an evolutionary process to escape mathematical plateaus without crossing its hidden boundaries. The network exchanges only compatible insights, strictly preserving each client’s unique geometry and data privacy. GeoEvo mathematically guarantees steady, safe improvement for the entire decentralized network. Extensive experiments show diverse devices learn together successfully, significantly boosting personalization and robustness under real-world data mismatches. Our work demonstrates that treating missing modalities as a geometric identity and modeling collaboration as a potential game fundamentally advances personalized multimodal federated learning.