CoBo: Collaborative Learning via Bilevel Optimization

Clients in collaborative learning aim to improve model quality through jointly training. Identifying helpful clients, however, presents challenging and often introduces significant overhead. In this paper, we model $\textit{client-selection}$ and $\textit{model-training}$ as two interconnected optimization problems, proposing a novel bilevel optimization problem for collaborative learning.We introduce CoBo, an efficient SGD-type alternating optimization algorithm that addresses collaborative learning with theoretical convergence guarantees. Moreover, CoBo presents strong empirical performances, outperforming all other algorithms in terms of model quality and fairness.