Timezone: »
Coding Theory For Large-scale Machine Learning
Coding theory involves the art and science of how to add redundancy to data to ensure that a desirable output is obtained at despite deviations from ideal behavior from the system components that interact with the data. Through a rich, mathematically elegant set of techniques, coding theory has come to significantly influence the design of modern data communications, compression and storage systems. The last few years have seen a rapidly growing interest in coding theory based approaches for the development of efficient machine learning algorithms towards robust, large-scale, distributed computational pipelines.
The CodML workshop brings together researchers developing coding techniques for machine learning, as well as researchers working on systems implementations for computing, with cutting-edge presentations from both sides. The goal is to learn about non-idealities in system components as well as approaches to obtain reliable and robust learning despite these non-idealities, and identify problems of future interest.
The workshop is co-located with ICML 2019, and will be held in Long Beach, California, USA on June 14th or 15th, 2019.
Please see the website for more details:
Call for Posters
Scope of the Workshop
In this workshop we solicit research papers focused on the application of coding and information-theoretic techniques for distributed machine learning. More broadly, we seek papers that address the problem of making machine learning more scalable, efficient, and robust. Both theoretical as well as experimental contributions are welcome. We invite authors to submit papers on topics including but not limited to:
- Asynchronous Distributed Training Methods
- Communication-Efficient Training
- Model Compression and Quantization
- Gradient Coding, Compression and Quantization
- Erasure Coding Techniques for Straggler Mitigation
- Data Compression in Large-scale Machine Learning
- Erasure Coding Techniques for ML Hardware Acceleration
- Fast, Efficient and Scalable Inference
- Secure and Private Machine Learning
- Data Storage/Access for Machine Learning Jobs
- Performance evaluation of coding techniques
Submission Format and Instructions
The authors should prepare extended abstracts in the ICML paper format and submit via CMT. Submitted papers may not exceed three (3) single-spaced double-column pages excluding references. All results, proofs, figures, tables must be included in the 3 pages. The submitted manuscripts should include author names and affiliations, and an abstract that does not exceed 250 words. The authors may include a link to an extended version of the paper that includes supplementary material (proofs, experimental details, etc.) but the reviewers are not required to read the extended version.
Dual Submission Policy
Accepted submissions will be considered non-archival and can be submitted elsewhere without modification, as long as the other conference allows it. Moreover, submissions to CodML based on work recently accepted to other venues are also acceptable (though authors should explicitly make note of this in their submissions).
Key Dates
Paper Submission: May 3rd, 2019, 11:59 PM anywhere on earth
Decision Notification: May 12th, 2019.
Workshop date: June 14 or 15, 2019
Sat 8:45 a.m. - 9:00 a.m.
|
Opening Remarks
|
🔗 |
Sat 9:00 a.m. - 9:30 a.m.
|
Salman Avestimehr: Lagrange Coded Computing: Optimal Design for Resilient, Secure, and Private Distributed Learning
(
Invited Talk
)
|
Salman Avestimehr 🔗 |
Sat 9:30 a.m. - 10:00 a.m.
|
Vivienne Sze: Exploiting redundancy for efficient processing of DNNs and beyond
(
Invited talk
)
|
Vivienne Sze 🔗 |
Sat 10:00 a.m. - 10:10 a.m.
|
"Locality Driven Coded Computation" Michael Rudow, Rashmi Vinayak and Venkat Guruswami
(
Spotlight Presentation
)
|
Michael Rudow 🔗 |
Sat 10:10 a.m. - 10:20 a.m.
|
"CodeNet: Training Large-Scale Neural Networks in Presence of Soft-Errors," Sanghamitra Dutta, Ziqian Bai, Tze Meng Low and Pulkit Grover
(
Spotlight Presentation
)
|
Sanghamitra Dutta 🔗 |
Sat 10:20 a.m. - 10:30 a.m.
|
"Reliable Clustering with Redundant Data Assignment" Venkat Gandikota, Arya Mazumdar and Ankit Singh Rawat
(
Spotlight Presentation
)
|
Venkata Gandikota 🔗 |
Sat 10:30 a.m. - 11:30 a.m.
|
Poster Session I
(
Poster Session
)
|
Stark Draper · Mehmet Aktas · Basak Guler · Hongyi Wang · Venkata Gandikota · Hyegyeong Park · Jinhyun So · Lev Tauz · hema venkata krishna giri Narra · Zhifeng Lin · Mohammadali Maddahali · Yaoqing Yang · Sanghamitra Dutta · Amirhossein Reisizadeh · Jianyu Wang · Eren Balevi · Siddharth Jain · Paul McVay · Michael Rudow · Pedro Soto · Jun Li · Adarsh Subramaniam · Umut Demirhan · Vipul Gupta · Deniz Oktay · Leighton P Barnes · Johannes BallĂ© · Farzin Haddadpour · Haewon Jeong · Rong-Rong Chen · Mohammad Fahim
|
Sat 11:30 a.m. - 12:00 p.m.
|
Rashmi Vinayak: Resilient ML inference via coded computation: A learning-based approach
(
Invited Talk
)
|
Rashmi Vinayak 🔗 |
Sat 12:00 p.m. - 1:30 p.m.
|
Lunch Break
|
🔗 |
Sat 1:30 p.m. - 2:00 p.m.
|
Markus Weimer: A case for coded computing on elastic compute
(
Invited Talk
)
|
Markus Weimer 🔗 |
Sat 2:00 p.m. - 2:30 p.m.
|
Alex Dimakis: Coding Theory for Distributed Learning
(
Invited Talk
)
|
Alexandros Dimakis 🔗 |
Sat 2:30 p.m. - 3:00 p.m.
|
Wei Zhang: Distributed deep learning system building at IBM: Scale-up and Scale-out case studies
(
Invited Talk
)
|
Wei Zhang 🔗 |
Sat 3:30 p.m. - 3:40 p.m.
|
"OverSketched Newton: Fast Convex Optimization for Serverless Systems," Vipul Gupta, Swanand Kadhe, Thomas Courtade, Michael Mahoney and Kannan Ramchandran
(
Spotlight Presentation
)
|
Vipul Gupta 🔗 |
Sat 3:40 p.m. - 3:50 p.m.
|
"Cooperative SGD: A Unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms", Jianyu Wang and Gauri Joshi
(
Spotlight Presentation
)
|
Jianyu Wang 🔗 |
Sat 3:50 p.m. - 4:00 p.m.
|
"Secure Coded Multi-Party Computation for Massive Matrices with Adversarial Nodes," Seyed Reza, Mohammad Ali Maddah-Ali and Mohammad Reza Aref
(
Spotlight Presentation
)
|
Mohammadali Maddahali 🔗 |
Sat 4:00 p.m. - 5:15 p.m.
|
Poster Session II
(
Poster Session
)
|
🔗 |
Author Information
Viveck Cadambe (The Pennsylvania State University)
Pulkit Grover (Carnegie Mellon University)
Dimitris Papailiopoulos (University of Wisconsin-Madison)
Gauri Joshi (Carnegie Mellon University)
More from the Same Authors
-
2021 : An Extreme Point Approach to Subset Selection »
Viveck Cadambe · Bill Kay -
2023 : Towards a Theoretical and Practical Understanding of One-Shot Federated Learning with Fisher Information »
Divyansh Jhunjhunwala · Shiqiang Wang · Gauri Joshi -
2023 : Teaching Arithmetic to Small Transformers »
Nayoung Lee · Kartik Sreenivasan · Jason Lee · Kangwook Lee · Dimitris Papailiopoulos -
2023 : Predictive Pipelined Decoding: A Compute-Latency Trade-off for Exact LLM Decoding »
Seongjun Yang · Gibbeum Lee · Jaewoong Cho · Dimitris Papailiopoulos · Kangwook Lee -
2023 : Looped Transformers are Better at Learning Learning Algorithms »
Liu Yang · Kangwook Lee · Robert Nowak · Dimitris Papailiopoulos -
2023 Poster: The Blessing of Heterogeneity in Federated Q-Learning: Linear Speedup and Beyond »
Jiin Woo · Gauri Joshi · Yuejie Chi -
2023 Poster: On the Convergence of Federated Averaging with Cyclic Client Participation »
Yae Jee Cho · PRANAY SHARMA · Gauri Joshi · Zheng Xu · Satyen Kale · Tong Zhang -
2023 Poster: Looped Transformers as Programmable Computers »
Angeliki Giannou · Shashank Rajput · Jy-yong Sohn · Kangwook Lee · Jason Lee · Dimitris Papailiopoulos -
2023 Poster: Transformers as Algorithms: Generalization and Stability in In-context Learning »
Yingcong Li · Muhammed Ildiz · Dimitris Papailiopoulos · Samet Oymak -
2022 Poster: Federated Reinforcement Learning: Linear Speedup Under Markovian Sampling »
sajad khodadadian · PRANAY SHARMA · Gauri Joshi · Siva Maguluri -
2022 Oral: Federated Reinforcement Learning: Linear Speedup Under Markovian Sampling »
sajad khodadadian · PRANAY SHARMA · Gauri Joshi · Siva Maguluri -
2022 Poster: GenLabel: Mixup Relabeling using Generative Models »
Jy yong Sohn · Liang Shang · Hongxu Chen · Jaekyun Moon · Dimitris Papailiopoulos · Kangwook Lee -
2022 Spotlight: GenLabel: Mixup Relabeling using Generative Models »
Jy yong Sohn · Liang Shang · Hongxu Chen · Jaekyun Moon · Dimitris Papailiopoulos · Kangwook Lee -
2022 Poster: Federated Minimax Optimization: Improved Convergence Analyses and Algorithms »
PRANAY SHARMA · Rohan Panda · Gauri Joshi · Pramod K Varshney -
2022 Spotlight: Federated Minimax Optimization: Improved Convergence Analyses and Algorithms »
PRANAY SHARMA · Rohan Panda · Gauri Joshi · Pramod K Varshney -
2021 : An Extreme Point Approach to Subset Selection »
Bill Kay · Viveck Cadambe -
2021 : Closing Remarks »
Shiqiang Wang · Nathalie Baracaldo · Olivia Choudhury · Gauri Joshi · Peter Richtarik · Praneeth Vepakomma · Han Yu -
2021 : Dreaming of Federated Robustness: Inherent Barriers and Unavoidable Tradeoffs »
Dimitris Papailiopoulos -
2021 Workshop: International Workshop on Federated Learning for User Privacy and Data Confidentiality in Conjunction with ICML 2021 (FL-ICML'21) »
Nathalie Baracaldo · Olivia Choudhury · Gauri Joshi · Peter Richtarik · Praneeth Vepakomma · Shiqiang Wang · Han Yu -
2021 : Opening Remarks »
Shiqiang Wang · Nathalie Baracaldo · Olivia Choudhury · Gauri Joshi · Peter Richtarik · Praneeth Vepakomma · Han Yu -
2020 : Closing remarks »
Nathalie Baracaldo · Olivia Choudhury · Gauri Joshi · Ramesh Raskar · Shiqiang Wang · Han Yu -
2020 : Opening remarks »
Nathalie Baracaldo · Olivia Choudhury · Gauri Joshi · Ramesh Raskar · Shiqiang Wang · Han Yu -
2020 Workshop: Federated Learning for User Privacy and Data Confidentiality »
Nathalie Baracaldo · Olivia Choudhury · Olivia Choudhury · Gauri Joshi · Ramesh Raskar · Gauri Joshi · Shiqiang Wang · Han Yu -
2020 Poster: Closing the convergence gap of SGD without replacement »
Shashank Rajput · Anant Gupta · Dimitris Papailiopoulos -
2019 Poster: Does Data Augmentation Lead to Positive Margin? »
Shashank Rajput · Zhili Feng · Zachary Charles · Po-Ling Loh · Dimitris Papailiopoulos -
2019 Poster: Trading Redundancy for Communication: Speeding up Distributed SGD for Non-convex Optimization »
Farzin Haddadpour · Mohammad Mahdi Kamani · Mehrdad Mahdavi · Viveck Cadambe -
2019 Oral: Does Data Augmentation Lead to Positive Margin? »
Shashank Rajput · Zhili Feng · Zachary Charles · Po-Ling Loh · Dimitris Papailiopoulos -
2019 Oral: Trading Redundancy for Communication: Speeding up Distributed SGD for Non-convex Optimization »
Farzin Haddadpour · Mohammad Mahdi Kamani · Mehrdad Mahdavi · Viveck Cadambe -
2018 Poster: DRACO: Byzantine-resilient Distributed Training via Redundant Gradients »
Lingjiao Chen · Hongyi Wang · Zachary Charles · Dimitris Papailiopoulos -
2018 Oral: DRACO: Byzantine-resilient Distributed Training via Redundant Gradients »
Lingjiao Chen · Hongyi Wang · Zachary Charles · Dimitris Papailiopoulos -
2018 Poster: Stability and Generalization of Learning Algorithms that Converge to Global Optima »
Zachary Charles · Dimitris Papailiopoulos -
2018 Oral: Stability and Generalization of Learning Algorithms that Converge to Global Optima »
Zachary Charles · Dimitris Papailiopoulos