Workshop

International Workshop on Federated Learning for User Privacy and Data Confidentiality in Conjunction with ICML 2021 (FL-ICML'21)

Nathalie Baracaldo, Olivia Choudhury, Gauri Joshi, Peter Richtarik, Praneeth Vepakomma, Shiqiang Wang, Han Yu

Abstract:

Training machine learning models in a centralized fashion often faces significant challenges due to regulatory and privacy concerns in real-world use cases. These include distributed training data, computational resources to create and maintain a central data repository, and regulatory guidelines (GDPR, HIPAA) that restrict sharing sensitive data. Federated learning (FL) is a new paradigm in machine learning that can mitigate these challenges by training a global model using distributed data, without the need for data sharing. The extensive application of machine learning to analyze and draw insight from real-world, distributed, and sensitive data necessitates familiarization with and adoption of this relevant and timely topic among the scientific community.

Despite the advantages of FL, and its successful application in certain industry-based cases, this field is still in its infancy due to new challenges that are imposed by limited visibility of the training data, potential lack of trust among participants training a single model, potential privacy inferences, and in some cases, limited or unreliable connectivity.

The goal of this workshop is to bring together researchers and practitioners interested in FL. This day-long event will facilitate interaction among students, scholars, and industry professionals from around the world to understand the topic, identify technical challenges, and discuss potential solutions. This will lead to an overall advancement of FL and its impact in the community, while noting that FL has become an increasingly popular topic in the ICML community in recent years.

Chat is not available.

Timezone: »

Schedule

Sat 5:00 a.m. - 5:15 a.m.
Opening Remarks
Shiqiang Wang, Nathalie Baracaldo, Olivia Choudhury, Gauri Joshi, Peter Richtarik, Praneeth Vepakomma, Han Yu
Sat 5:15 a.m. - 5:40 a.m.
Algorithms for Efficient Federated and Decentralized Learning (Invited Talk)   
Sebastian Stich
Sat 5:40 a.m. - 5:45 a.m.
Algorithms for Efficient Federated and Decentralized Learning (Q&A) (Q&A)
Sebastian Stich
Sat 5:45 a.m. - 6:45 a.m.
Contributed Oral Presentation Session 1 (Live Presentations)   
Sat 6:45 a.m. - 7:10 a.m.
The ML data center is dead: What comes next? (Invited Talk)
Nic Lane
Sat 7:10 a.m. - 7:15 a.m.
The ML data center is dead: What comes next? (Q&A) (Q&A)
Nic Lane
Sat 7:15 a.m. - 7:30 a.m.
Break
Sat 7:30 a.m. - 8:00 a.m.
Contributed Oral Presentation Session 2 (Live Presentations)   
Sat 8:00 a.m. - 8:25 a.m.
Pandemic Response with Crowdsourced Data: The Participatory Privacy Preserving Approach (Invited Talk)   
Ramesh Raskar
Sat 8:25 a.m. - 8:30 a.m.
Pandemic Response with Crowdsourced Data: The Participatory Privacy Preserving Approach (Q&A) (Q&A)
Ramesh Raskar
Sat 8:30 a.m. - 9:30 a.m.
Industrial Panel (Panel)   
Nathalie Baracaldo, Shiqiang Wang, Peter Kairouz, Zheng Xu, Kshitiz Malik, Tao Zhang
Sat 9:30 a.m. - 10:30 a.m.
Poster Session 1 & Industrial Booths (GatherTown Session)  link »
Sat 10:30 a.m. - 11:00 a.m.
Break
Sat 11:00 a.m. - 11:25 a.m.
Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing (Invited Talk)   
Ameet Talwalkar
Sat 11:25 a.m. - 11:30 a.m.
Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing (Q&A) (Q&A)
Ameet Talwalkar
Sat 11:30 a.m. - 12:30 p.m.
Contributed Oral Presentation Session 3 (Live Presentations)   
Sat 12:30 p.m. - 12:55 p.m.
Optimization Aspects of Personalized Federated Learning (Invited Talk)   
Filip Hanzely
Sat 12:55 p.m. - 1:00 p.m.
Optimization Aspects of Personalized Federated Learning (Q&A) (Q&A)
Filip Hanzely
Sat 1:00 p.m. - 2:00 p.m.
Poster Session 2 & Industrial Booths (GatherTown Session)  link »
Sat 2:00 p.m. - 2:15 p.m.
Break
Sat 2:15 p.m. - 2:40 p.m.
Dreaming of Federated Robustness: Inherent Barriers and Unavoidable Tradeoffs (Invited Talk)   
Dimitris Papailiopoulos
Sat 2:40 p.m. - 2:45 p.m.
Dreaming of Federated Robustness: Inherent Barriers and Unavoidable Tradeoffs (Q&A) (Q&A)
Sat 2:45 p.m. - 3:10 p.m.
  

Secure aggregation is a critical component in federated learning, which enables the server to learn the aggregate model of the users without observing their local models. Conventionally, secure aggregation algorithms focus only on ensuring theprivacy of individual users in a single training round. We contend that such designs can lead to significant privacy leakages over multiple training rounds, due to partial user selection/participation at each round of federated learning. In fact, we empiricallyshow that the conventional random user selection strategies for federated learning lead to leaking users' individual models within number of rounds linear in the number of users. To address this challenge, we introduce a secure aggregation framework with multi-roundprivacy guarantees. In particular, we introduce a new metric to quantify the privacy guarantees of federated learning over multiple training rounds, and develop a structured user selection strategy that guarantees the long-term privacy of each user (over anynumber of training rounds). Our framework also carefully accounts for the fairness and the average number of participating users at each round. We perform several experiments on various datasets in the IID and the non-IID settings to demonstrate the performanceimprovement over the baseline algorithms, both in terms of privacy protection and test accuracy. We conclude the talk by discussing several open problems in this domain. (This talk is based on the following paper: https://arxiv.org/abs/2106.03328)

Salman Avestimehr
Sat 3:10 p.m. - 3:15 p.m.
Securing Secure Aggregation: Mitigating Multi-Round Privacy Leakage in Federated Learning (Q&A) (Q&A)
Salman Avestimehr
Sat 3:15 p.m. - 3:30 p.m.
Closing Remarks   
Shiqiang Wang, Nathalie Baracaldo, Olivia Choudhury, Gauri Joshi, Peter Richtarik, Praneeth Vepakomma, Han Yu
-
Implicit Gradient Alignment in Distributed and Federated Learning (Workshop Poster) [ Visit Poster at Spot C1 in Virtual World ]
-
Decentralized federated learning of deep neural networks on non-iid data (Workshop Poster) [ Visit Poster at Spot B1 in Virtual World ]
-
GRP-FED: Addressing Client Imbalance in Federated Learning via Global-Regularized Personalization (Workshop Poster) [ Visit Poster at Spot C4 in Virtual World ]
-
Accelerating Federated Learning with Split Learning on Locally Generated Losses (Workshop Poster) [ Visit Poster at Spot A6 in Virtual World ]
-
Handling Both Stragglers and Adversaries for Robust Federated Learning (Workshop Poster) [ Visit Poster at Spot C5 in Virtual World ]
-
Towards Federated Learning With Byzantine-Robust Client Weighting (Workshop Poster) [ Visit Poster at Spot D2 in Virtual World ]
-
SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networks (Workshop Poster) [ Visit Poster at Spot D0 in Virtual World ]
-
Defending against Reconstruction Attack in Vertical Federated Learning (Workshop Poster) [ Visit Poster at Spot B2 in Virtual World ]
-
Fed-EINI: An Efficient and Interpretable Inference Framework for Decision Tree Ensembles in Federated Learning (Workshop Poster) [ Visit Poster at Spot B3 in Virtual World ]
-
FlyNN: Fruit-fly Inspired Federated Nearest Neighbor Classification (Workshop Poster) [ Visit Poster at Spot C0 in Virtual World ]
-
FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout (Workshop Poster) [ Visit Poster at Spot C2 in Virtual World ]
-
MURANA: A Generic Framework for Stochastic Variance-Reduced Optimization (Workshop Poster) [ Visit Poster at Spot D0 in Virtual World ]
-
Smoothness-Aware Quantization Techniques (Workshop Poster) [ Visit Poster at Spot D3 in Virtual World ]
-
FedNL: Making Newton-Type Methods Applicable to Federated Learning (Workshop Poster) [ Visit Poster at Spot C1 in Virtual World ]
-
OmniLytics: A Blockchain-based Secure Data Market for Decentralized Machine Learning (Workshop Poster) [ Visit Poster at Spot D1 in Virtual World ]
-
Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneity (Workshop Poster) [ Visit Poster at Spot D1 in Virtual World ]
-
Federated Random Reshuffling with Compression and Variance Reduction (Workshop Poster) [ Visit Poster at Spot B6 in Virtual World ]
-
Federated Learning with Metric Loss (Workshop Poster) [ Visit Poster at Spot B4 in Virtual World ]
-
FedMix: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learning (Workshop Poster) [ Visit Poster at Spot B6 in Virtual World ]
-
Achieving Optimal Sample and Communication Complexities for Non-IID Federated Learning (Workshop Poster) [ Visit Poster at Spot A4 in Virtual World ]
-
A New Analysis Framework for Federated Learning on Time-Evolving Heterogeneous Data (Workshop Poster) [ Visit Poster at Spot A4 in Virtual World ]
-
Subgraph Federated Learning with Missing Neighbor Generation (Workshop Poster) [ Visit Poster at Spot D4 in Virtual World ]
-
Byzantine Fault-Tolerance of Local Gradient-Descent in Federated Model under 2f-Redundancy (Workshop Poster) [ Visit Poster at Spot B0 in Virtual World ]
-
Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy (Workshop Poster) [ Visit Poster at Spot D3 in Virtual World ]
-
Federated Graph Classification over Non-IID Graphs (Workshop Poster) [ Visit Poster at Spot B4 in Virtual World ]
-
Local Adaptivity in Federated Learning: Convergence and Consistency (Workshop Poster) [ Visit Poster at Spot C2 in Virtual World ]
-
New Metrics to Evaluate the Performance and Fairness of Personalized Federated Learning (Workshop Poster) [ Visit Poster at Spot C4 in Virtual World ]
-
Diverse Client Selection for Federated Learning: Submodularity and Convergence Analysis (Workshop Poster) [ Visit Poster at Spot B3 in Virtual World ]
-
BYGARS: Byzantine SGD with Arbitrary Number of Attackers Using Reputation Scores (Workshop Poster) [ Visit Poster at Spot A6 in Virtual World ]
-
Bi-directional Adaptive Communication for Heterogenous Distributed Learning (Workshop Poster) [ Visit Poster at Spot A5 in Virtual World ]
-
Communication and Energy Efficient Slimmable Federated Learning via Superposition Coding and Successive Decoding (Workshop Poster) [ Visit Poster at Spot B1 in Virtual World ]
-
BiG-Fed: Bilevel Optimization Enhanced Graph-Aided Federated Learning (Workshop Poster) [ Visit Poster at Spot B0 in Virtual World ]
-
Gradient Inversion with Generative Image Prior (Workshop Poster) [ Visit Poster at Spot C3 in Virtual World ]
-
On Large-Cohort Training for Federated Learning (Workshop Poster) [ Visit Poster at Spot C5 in Virtual World ]
-
FedGNN: Federated Graph Neural Network for Privacy-Preserving Recommendation (Workshop Poster) [ Visit Poster at Spot C0 in Virtual World ]
-
Robust and Differentially Private Mean Estimation (Workshop Poster) [ Visit Poster at Spot C6 in Virtual World ]
-
A Reputation Mechanism Is All You Need: Collaborative Fairness and Adversarial Robustness in Federated Learning (Workshop Poster) [ Visit Poster at Spot A5 in Virtual World ]
-
Multistage stepsize schedule in Federated Learning: Bridging Theory and Practice (Workshop Poster) [ Visit Poster at Spot C3 in Virtual World ]
-
Lower Bounds and Optimal Algorithms for Smooth and Strongly Convex Decentralized Optimization over Time-Varying Networks (Workshop Poster) [ Visit Poster at Spot C6 in Virtual World ]
-
Federated Multi-Task Learning under a Mixture of Distributions (Workshop Poster) [ Visit Poster at Spot B5 in Virtual World ]
-
EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedback (Workshop Poster) [ Visit Poster at Spot B2 in Virtual World ]
-
Optimal Model Averaging: Towards Personalized Collaborative Learning (Workshop Poster) [ Visit Poster at Spot D2 in Virtual World ]
-
Federated Learning with Buffered Asynchronous Aggregation (Workshop Poster) [ Visit Poster at Spot B5 in Virtual World ]
-
Industrial Booth (IBM) (Workshop Poster) [ Visit Poster at Spot A3 in Virtual World ]
-
Industrial Booth (Facebook) (Workshop Poster) [ Visit Poster at Spot A1 in Virtual World ]
-
Industrial Booth (Google) (Workshop Poster) [ Visit Poster at Spot A2 in Virtual World ]
-
- (Workshop Poster) [ Visit Poster at Spot A0 in Virtual World ]