Towards Rule-Based Knowledge Sharing in Federated Learning

Federated learning often face both data and model heterogeneity, with the latter often more challenging. Architectural differences yield incompatible representation, making the knowledge-sharing carrier central to heterogeneous collaboration. Using proxy model enables distillation-based collaboration but incurs high communication and computation costs. Prototype-based carriers are lighter yet cause semantic confusion when incompatible features are mixed. Therefore, we propose rule-based federated learning (RFL) that shares interpretable, class-discriminative rules to enable heterogeneous collaboration, avoid feature confusion, and keep communication lightweight. RFL uses a rule network to unify clients’ decision features and collaborates at the rule level, avoiding forcible averaging of incompatible representations. RFL selects sparse, high-coverage, beneficial rules for broadcasting, compressing shared knowledge into an interpretable class-rule set and reducing communication and computation costs. Each client selectively activates only rules relevant to its local classes, mitigating negative transfer while preserving personalization. Across heterogeneous settings, RFL achieves a better accuracy–communication trade-off.

Federated learning allows multiple clients to collaboratively train models without sharing raw data. In practice, clients may have different data distributions and resource environments, and therefore use models with different architectures and sizes. This raises a key question: when different models learn from and represent data differently, what knowledge should they share? Exchanging intermediate features may mix semantically incompatible information, causing feature confusion and misleading collaboration, and may instead weaken collaboration. This paper proposes RFL, which enables models with different architectures to share knowledge in a stable and understandable way. Instead of exchanging hard-to-align features, RFL transforms what clients have learned about different classes into compact and interpretable class decision rules. The server collects class summaries uploaded by clients, extracts useful decision rules from them, and sends these rules back to clients. Each client only absorbs rules related to its own task, reducing the influence of irrelevant or harmful information. In this way, RFL turns collaboration among heterogeneous models from feature exchange into rule sharing. Experiments show that RFL improves collaboration among different models while reducing communication and computation costs and maintaining interpretability. Therefore, RFL provides a lightweight, interpretable, and effective collaboration approach for heterogeneous federated learning.