Timezone: »

 
Poster
Federated Learning with Only Positive Labels
Felix Xinnan Yu · Ankit Singh Rawat · Aditya Menon · Sanjiv Kumar

Tue Jul 14 10:00 AM -- 10:45 AM & Tue Jul 14 11:00 PM -- 11:45 PM (PDT) @
in Poster Session 4 »

We consider learning a multi-class classification model in the federated setting, where each user has access to the positive data associated with only a single class. As a result, during each federated learning round, the users need to locally update the classifier without having access to the features and the model parameters for the negative classes. Thus, naively employing conventional decentralized learning such as distributed SGD or Federated Averaging may lead to trivial or extremely poor classifiers. In particular, for embedding based classifiers, all the class embeddings might collapse to a single point. To address this problem, we propose a generic framework for training with only positive labels, namely Federated Averaging with Spreadout (FedAwS), where the server imposes a geometric regularizer after each round to encourage classes to be spreadout in the embedding space. We show, both theoretically and empirically, that FedAwS can almost match the performance of conventional learning where users have access to negative labels. We further extend the proposed method to settings with large output spaces.

Keywords: Learning Theory · Parallel and Distributed Learning · Privacy-preserving Statistics and Machine Learning · Supervised Learning

Author Information

Felix Xinnan Yu (Google)
Ankit Singh Rawat (Google)
Aditya Menon (Google Research)
Sanjiv Kumar (Google Research, NY)

More from the Same Authors