Poster

Federated Learning with Only Positive Labels

Felix Xinnan Yu · Ankit Singh Rawat · Aditya Menon · Sanjiv Kumar

Keywords: [ Learning Theory ] [ Parallel and Distributed Learning ] [ Privacy-preserving Statistics and Machine Learning ] [ Supervised Learning ]

[ Abstract ] [ Join Zoom
Please do not share or post zoom links

Abstract:

We consider learning a multi-class classification model in the federated setting, where each user has access to the positive data associated with only a single class. As a result, during each federated learning round, the users need to locally update the classifier without having access to the features and the model parameters for the negative classes. Thus, naively employing conventional decentralized learning such as distributed SGD or Federated Averaging may lead to trivial or extremely poor classifiers. In particular, for embedding based classifiers, all the class embeddings might collapse to a single point. To address this problem, we propose a generic framework for training with only positive labels, namely Federated Averaging with Spreadout (FedAwS), where the server imposes a geometric regularizer after each round to encourage classes to be spreadout in the embedding space. We show, both theoretically and empirically, that FedAwS can almost match the performance of conventional learning where users have access to negative labels. We further extend the proposed method to settings with large output spaces.

Chat is not available.