Abstract: Clients in collaborative learning aim to improve model quality through jointly training. Identifying helpful clients, however, presents challenging and often introduces significant overhead. In this paper, we model $\textit{client-selection}$ and $\textit{model-training}$ as two interconnected optimization problems, proposing a novel bilevel optimization problem for collaborative learning.We introduce CoBo, an efficient SGD-type alternating optimization algorithm that addresses collaborative learning with theoretical convergence guarantees. Moreover, CoBo presents strong empirical performances, outperforming all other algorithms in terms of model quality and fairness.