The Hidden Risk: Membership Inference Attacks on Multimodal Federated Learning via Modality Imbalance

Federated learning (FL) faces significant challenges from modality heterogeneity, which motivates multimodal federated learning (MFL) to leverage complementary modalities across decentralized clients for improved performance. However, modality imbalance introduces a new attack surface, making MFL more vulnerable to membership inference attacks (MIAs), an issue that remains largely unexplored. In this work, we present the first systematic study of MIAs against MFL and propose a modality-aware attack framework. We show that multimodal models are inherently more susceptible to MIAs due to heterogeneous modality contributions, and existing attacks are suboptimal as they treat multimodal parameters as a whole. By performing MIAs on individual modalities, we find that (i) attacking the dominant modality achieves comparable accuracy with lower overhead, and (ii) different modalities expose distinct membership patterns. To identify members with different patterns, we propose a modality-aware framework that exploits cross-modal performance gaps to adaptively select attack modalities and calibrate inference results. Experiments on three datasets show our approach outperforms baselines across multiple metrics.

Just as most people prefer using their right hand over their left, making it more skilled through repeated use, multimodal models tend to rely more heavily on one data modality than others. This imbalance is common but often overlooked. We discovered that this "dominant modality" phenomenon introduces a hidden privacy risk in federated learning, a framework designed to train AI models without directly sharing user data. Because the model depends more on certain modalities, those modalities are more likely to leak private information about whether a specific individual's data was used during training. This threat is known as a membership inference attack. Our findings reveal a previously overlooked privacy vulnerability in multimodal federated learning. This warns researchers and practitioners to account for modality imbalance when designing privacy-preserving AI systems.