FedECADO: A Dynamical System Model of Federated Learning

Federated learning harnesses the power of distributed optimization to train a unified machine learning model across separate clients. However, heterogeneous data distributions and computational workloads can lead to inconsistent updates and limit model performance. This work tackles these challenges by proposing FedECADO, a new algorithm inspired by a dynamical system representation of the federated learning process. FedECADO addresses non-IID data distribution through an aggregate sensitivity model that reflects the amount of data processed by each client. To tackle heterogeneous computing, we design a multi-rate integration method with adaptive step-size selections that synchronizes active client updates in continuous time. Compared to prominent techniques, including FedProx, FedExp, and FedNova, FedECADO achieves higher classification accuracies in numerous heterogeneous scenarios.

As machine learning models and their training datasets get bigger, it is no longer practical to train them on just one computer. Federated learning helps by dividing the training process across many computers, each working with its own slice of data. However, this setup brings new challenges where not all computers have the same computational abilities and the data they use can vary widely. To address this, we created a new method to train models across multiple machines while accounting for these computational differences. Our method draws on principles from electrical circuits, enabling us to design faster and more efficient ways to train machine learning models at scale.