Sparsity in Deep Learning: Pruning and growth for efficient inference and training

Torsten Hoefler · Dan Alistarh


This tutorial will perform an detailed overview of the work on sparsity in deep learning, covering sparsifi- cation techniques for neural networks, from both the mathematical and implementation perspectives. We specifically aim to cover the significant recent advances in the area, and put them in the context of the foundational work performed on this topic in the 1990s.

Chat is not available.

Mon 8:00 a.m. - 8:40 a.m.
Part 1: Introduction to Sparsity in Deep Learning (Presentation)   
Torsten Hoefler
Mon 8:45 a.m. - 9:30 a.m.
Part 2: Mathematical Background (Presentation)   
Dan Alistarh
Mon 9:30 a.m. - 10:00 a.m.
Part 3: Case study on CNN sparsification on ImageNet (Presentation)   
Dan Alistarh
Mon 10:00 a.m. - 10:40 a.m.
Part 4: Ephemeral Sparsity, Sparse Training, and Conclusions (Presentation)   
Torsten Hoefler
Mon 10:45 a.m. - 10:59 a.m.
Extended Q&A (Discussion)