Timezone: »

Coding Theory For Large-scale Machine Learning
Viveck Cadambe · Pulkit Grover · Dimitris Papailiopoulos · Gauri Joshi

Sat Jun 15 08:30 AM -- 06:00 PM (PDT) @ 202

Coding Theory For Large-scale Machine Learning

Coding theory involves the art and science of how to add redundancy to data to ensure that a desirable output is obtained at despite deviations from ideal behavior from the system components that interact with the data. Through a rich, mathematically elegant set of techniques, coding theory has come to significantly influence the design of modern data communications, compression and storage systems. The last few years have seen a rapidly growing interest in coding theory based approaches for the development of efficient machine learning algorithms towards robust, large-scale, distributed computational pipelines.

The CodML workshop brings together researchers developing coding techniques for machine learning, as well as researchers working on systems implementations for computing, with cutting-edge presentations from both sides. The goal is to learn about non-idealities in system components as well as approaches to obtain reliable and robust learning despite these non-idealities, and identify problems of future interest.

The workshop is co-located with ICML 2019, and will be held in Long Beach, California, USA on June 14th or 15th, 2019.

Please see the website for more details:

Call for Posters

Scope of the Workshop

In this workshop we solicit research papers focused on the application of coding and information-theoretic techniques for distributed machine learning. More broadly, we seek papers that address the problem of making machine learning more scalable, efficient, and robust. Both theoretical as well as experimental contributions are welcome. We invite authors to submit papers on topics including but not limited to:

  • Asynchronous Distributed Training Methods
  • Communication-Efficient Training
  • Model Compression and Quantization
  • Gradient Coding, Compression and Quantization
  • Erasure Coding Techniques for Straggler Mitigation
  • Data Compression in Large-scale Machine Learning
  • Erasure Coding Techniques for ML Hardware Acceleration
  • Fast, Efficient and Scalable Inference
  • Secure and Private Machine Learning
  • Data Storage/Access for Machine Learning Jobs
  • Performance evaluation of coding techniques

Submission Format and Instructions

The authors should prepare extended abstracts in the ICML paper format and submit via CMT. Submitted papers may not exceed three (3) single-spaced double-column pages excluding references. All results, proofs, figures, tables must be included in the 3 pages. The submitted manuscripts should include author names and affiliations, and an abstract that does not exceed 250 words. The authors may include a link to an extended version of the paper that includes supplementary material (proofs, experimental details, etc.) but the reviewers are not required to read the extended version.

Dual Submission Policy

Accepted submissions will be considered non-archival and can be submitted elsewhere without modification, as long as the other conference allows it. Moreover, submissions to CodML based on work recently accepted to other venues are also acceptable (though authors should explicitly make note of this in their submissions).

Key Dates

Paper Submission: May 3rd, 2019, 11:59 PM anywhere on earth

Decision Notification: May 12th, 2019.

Workshop date: June 14 or 15, 2019

Sat 8:45 a.m. - 9:00 a.m.
Opening Remarks
Sat 9:00 a.m. - 9:30 a.m.
Salman Avestimehr: Lagrange Coded Computing: Optimal Design for Resilient, Secure, and Private Distributed Learning (Invited Talk) [ Video
Salman Avestimehr
Sat 9:30 a.m. - 10:00 a.m.
Vivienne Sze: Exploiting redundancy for efficient processing of DNNs and beyond (Invited talk) [ Video
Vivienne Sze
Sat 10:00 a.m. - 10:10 a.m.
"Locality Driven Coded Computation" Michael Rudow, Rashmi Vinayak and Venkat Guruswami (Spotlight Presentation) [ Video
Michael Rudow
Sat 10:10 a.m. - 10:20 a.m.
"CodeNet: Training Large-Scale Neural Networks in Presence of Soft-Errors," Sanghamitra Dutta, Ziqian Bai, Tze Meng Low and Pulkit Grover (Spotlight Presentation) [ Video
Sanghamitra Dutta
Sat 10:20 a.m. - 10:30 a.m.
"Reliable Clustering with Redundant Data Assignment" Venkat Gandikota, Arya Mazumdar and Ankit Singh Rawat (Spotlight Presentation) [ Video
Venkata Gandikota
Sat 10:30 a.m. - 11:30 a.m.
Poster Session I (Poster Session)
Stark Draper, Mehmet Aktas, Basak Guler, Hongyi Wang, Venkata Gandikota, Hyegyeong Park, Jinhyun So, Lev Tauz, Krishna Narra, Zhifeng Lin, Mohammadali Maddahali, Yaoqing Yang, Sanghamitra Dutta, Amirhossein Reisizadeh, Jianyu Wang, Eren Balevi, Siddharth Jain Jain, Paul McVay, Michael Rudow, Pedro Soto, Jun Li, Adarsh Subramaniam, Umut Demirhan, Vipul Gupta, Deniz Oktay, Leighton P Barnes, Johannes Ballé, Farzin Haddadpour, Haewon Jeong, Rong-Rong Chen, Mohammad Fahim
Sat 11:30 a.m. - 12:00 p.m.
Rashmi Vinayak: Resilient ML inference via coded computation: A learning-based approach (Invited Talk) [ Video
Rashmi Vinayak
Sat 12:00 p.m. - 1:30 p.m.
Lunch Break (Break)
Sat 1:30 p.m. - 2:00 p.m.
Markus Weimer: A case for coded computing on elastic compute (Invited Talk) [ Video
Markus Weimer
Sat 2:00 p.m. - 2:30 p.m.
Alex Dimakis: Coding Theory for Distributed Learning (Invited Talk) [ Video
Alex Dimakis
Sat 2:30 p.m. - 3:00 p.m.
Wei Zhang: Distributed deep learning system building at IBM: Scale-up and Scale-out case studies (Invited Talk) [ Video
Wei Zhang
Sat 3:30 p.m. - 3:40 p.m.
"OverSketched Newton: Fast Convex Optimization for Serverless Systems," Vipul Gupta, Swanand Kadhe, Thomas Courtade, Michael Mahoney and Kannan Ramchandran (Spotlight Presentation) [ Video
Vipul Gupta
Sat 3:40 p.m. - 3:50 p.m.
"Cooperative SGD: A Unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms", Jianyu Wang and Gauri Joshi (Spotlight Presentation) [ Video
Jianyu Wang
Sat 3:50 p.m. - 4:00 p.m.
"Secure Coded Multi-Party Computation for Massive Matrices with Adversarial Nodes," Seyed Reza, Mohammad Ali Maddah-Ali and Mohammad Reza Aref (Spotlight Presentation) [ Video
Mohammadali Maddahali
Sat 4:00 p.m. - 5:15 p.m.
Poster Session II (Poster Session)

Author Information

Viveck Cadambe (The Pennsylvania State University)
Pulkit Grover (Carnegie Mellon University)
Dimitris Papailiopoulos (University of Wisconsin-Madison)
Gauri Joshi (Carnegie Mellon University)

More from the Same Authors