Timezone: »
This joint workshop aims to bring together researchers, educators, practitioners who are interested in techniques as well as applications of on-device machine learning and compact, efficient neural network representations. One aim of the workshop discussion is to establish close connection between researchers in the machine learning community and engineers in industry, and to benefit both academic researchers as well as industrial practitioners. The other aim is the evaluation and comparability of resource-efficient machine learning methods and compact and efficient network representations, and their relation to particular target platforms (some of which may be highly optimized for neural network inference). The research community has still to develop established evaluation procedures and metrics.
The workshop also aims at reproducibility and comparability of methods for compact and efficient neural network representations, and on-device machine learning. Contributors are thus encouraged to make their code available. The workshop organizers plan to make some example tasks and datasets available, and invite contributors to use them for testing their work. In order to provide comparable performance evaluation conditions, the use of a common platform (such as Google Colab) is intended.
|
Fri 8:30 a.m. - 8:40 a.m.
|
Welcome and Introduction
(
Talk
)
|
🔗 |
|
Fri 8:40 a.m. - 9:10 a.m.
|
Hardware Efficiency Aware Neural Architecture Search and Compression
(
Invited talk
)
|
Song Han 🔗 |
|
Fri 9:10 a.m. - 9:40 a.m.
|
Structured matrices for efficient deep learning
(
Invited talk
)
|
Sanjiv Kumar 🔗 |
|
Fri 9:40 a.m. - 10:00 a.m.
|
DeepCABAC: Context-adaptive binary arithmetic coding for deep neural network compression
(
Talk
)
Simon Wiedemann, Heiner Kirchhoffer, Stefan Matlage, Paul Haase, Arturo Marban Gonzalez, Talmaj Marinc, Heiko Schwarz, Detlev Marpe, Thomas Wiegand, Ahmed Osman and Wojciech Samek http://arxiv.org/abs/1905.08318 |
Simon Wiedemann 🔗 |
|
Fri 10:00 a.m. - 10:30 a.m.
|
Poster spotlight presentations
(
Talk
)
2min presentations of the posters presentations during lunch break |
🔗 |
|
Fri 10:30 a.m. - 11:00 a.m.
|
Coffee Break AM
|
🔗 |
|
Fri 11:00 a.m. - 11:30 a.m.
|
Understanding the Challenges of Algorithm and Hardware Co-design for Deep Neural Networks
(
Invited talk
)
The co-design of algorithm and hardware has become an increasingly important approach for addressing the computational complexity of Deep Neural Networks (DNNs). There are several open problems and challenges in the co-design process and application; for instance, what metrics should be used to drive the algorithm design, how to automate the process in a simple way, how to extend these approaches to tasks beyond image classification, and how to design flexible hardware to support these different approaches. In this talk, we highlight recent and ongoing work that aim to address these challenges, namely energy-aware pruning and NetAdapt that automatically incorporate direct metrics such as latency and energy into the training and design of the DNN; FastDepth that extends the co-design approaches to a depth estimation task; and a flexible hardware accelerator called Eyeriss v2 that is computationally efficient across a wide range of diverse DNNs.
BIO: Vivienne Sze is an Associate Professor at MIT in the Electrical Engineering and Computer Science Department. Her research interests include energy-aware signal processing algorithms, and low-power circuit and system design for portable multimedia applications, including computer vision, deep learning, autonomous navigation, and video process/coding. Prior to joining MIT, she was a Member of Technical Staff in the R&D Center at TI, where she designed low-power algorithms and architectures for video coding. She also represented TI in the JCT-VC committee of ITU-T and ISO/IEC standards body during the development of High Efficiency Video Coding (HEVC), which received a Primetime Engineering Emmy Award. She is a co-editor of the book entitled “High Efficiency Video Coding (HEVC): Algorithms and Architectures” (Springer, 2014). |
Vivienne Sze 🔗 |
|
Fri 11:30 a.m. - 11:50 a.m.
|
Dream Distillation: A Data-Independent Model Compression Framework
(
Talk
)
Kartikeya Bhardwaj, Naveen Suda and Radu Marculescu http://arxiv.org/abs/1905.07072 |
Kartikeya Bhardwaj 🔗 |
|
Fri 11:50 a.m. - 12:10 p.m.
|
The State of Sparsity in Deep Neural Networks
(
Talk
)
Trevor Gale, Erich Elsen and Sara Hooker https://arxiv.org/abs/1902.09574 (to be updated) |
Trevor Gale 🔗 |
|
Fri 12:10 p.m. - 12:40 p.m.
|
Lunch break
|
🔗 |
|
Fri 12:40 p.m. - 2:00 p.m.
|
Poster session
(
Posters
)
Xiaofan Zhang, Hao Cong, Yuhong Li, Yao Chen, Jinjun Xiong, Wen-Mei Hwu and Deming Chen. A Bi-Directional Co-Design Approach to Enable Deep Learning on IoT Devices https://arxiv.org/abs/1905.08369 Kushal Datta, Aishwarya Bhandare, Deepthi Karkada, Vamsi Sripathi, Sun Choi, Vikram Saletore and Vivek Menon. Efficient 8-Bit Quantization of Transformer Neural Machine Language Translation Model Bin Yang, Lin Yang, Xiaochun Li, Wenhan Zhang, Hua Zhou, Yequn Zhang, Yongxiong Ren and Yinbo Shi. 2-bit Model Compression of Deep Convolutional Neural Network on ASIC Engine for image retrieval https://arxiv.org/abs/1905.03362 Zhong Qiu Lin, Brendan Chwyl and Alexander Wong. EdgeSegNet: A Compact Network for Semantic Segmentation https://arxiv.org/abs/1905.04222 Sheng Lin, Xiaolong Ma, Shaokai Ye, Geng Yuan, Kaisheng Ma and Yanzhi Wang. Toward Extremely Low Bit and Lossless Accuracy in DNNs with Progressive ADMM https://arxiv.org/abs/1905.00789 Chengcheng Li, Zi Wang, Dali Wang, Xiangyang Wang and Hairong Qi. Investigating Channel Pruning through Structural Redundancy Reduction - A Statistical Study https://arxiv.org/abs/1905.06498 Wei Niu, Yanzhi Wang and Bin Ren. CADNN: Ultra Fast Execution of DNNs on Mobile Devices with Advanced Model Compression and Architecture-Aware Optimization https://arxiv.org/abs/1905.00571 Jonathan Ephrath, Lars Ruthotto, Eldad Haber and Eran Treister. LeanResNet: A Low-cost yet Effective Convolutional Residual Networks https://arxiv.org/abs/1904.06952 Dushyant Mehta, Kwang In Kim and Christian Theobalt. Implicit Filter Sparsification In Convolutional Neural Networks https://arxiv.org/abs/1905.04967 |
Cong Hao · Zhongqiu Lin · Chengcheng Li · Lars Ruthotto · Bin Yang · Deepthi Karkada 🔗 |
|
Fri 2:00 p.m. - 2:30 p.m.
|
DNN Training and Inference with Hyper-Scaled Precision
(
Invited talk
)
Kailash Gopalakrishnan |
Kailash Gopalakrishnan 🔗 |
|
Fri 2:30 p.m. - 3:00 p.m.
|
Mixed Precision Training & Inference
(
Invited talk
)
|
Jonathan Dekhtiar 🔗 |
|
Fri 3:00 p.m. - 3:30 p.m.
|
Coffee Break PM
|
🔗 |
|
Fri 3:30 p.m. - 3:50 p.m.
|
Learning Compact Neural Networks Using Ordinary Differential Equations as Activation Functions
(
Talk
)
Mohamadali Torkamani, Phillip Wallis, Shiv Shankar and Amirmohammad Rooshenas https://arxiv.org/abs/1905.07685 |
🔗 |
|
Fri 3:50 p.m. - 4:10 p.m.
|
Triplet Distillation for Deep Face Recognition
(
Talk
)
Yushu Feng, Huan Wang and Haoji Hu https://arxiv.org/abs/1905.04457 |
🔗 |
|
Fri 4:10 p.m. - 4:30 p.m.
|
Single-Path NAS: Device-Aware Efficient ConvNet Design
(
Talk
)
Dimitrios Stamoulis, Ruizhou Ding, Di Wang, Dimitrios Lymberopoulos, Bodhi Priyantha, Jie Liu and Diana Marculescu https://arxiv.org/abs/1905.04159 |
Dimitrios Stamoulis 🔗 |
|
Fri 4:30 p.m. - 5:30 p.m.
|
Panel discussion
(
Discussion
)
|
🔗 |
|
Fri 5:30 p.m. - 5:45 p.m.
|
Wrap-up and Closing
(
Talk
)
|
🔗 |
Author Information
Sujith Ravi (Google Research)
Zornitsa Kozareva (Google)
Lixin Fan (JD.com)
Max Welling (University of Amsterdam & Qualcomm)
Yurong Chen (Intel)
Werner Bailer (JOANNEUM RESEARCH)
Brian Kulis (Boston University)
Haoji Hu (Zhejiang University)
Jonathan Dekhtiar (NVIDIA)
Yingyan Lin (Rice University)
Diana Marculescu (Carnegie Mellon University)
More from the Same Authors
-
2022 : Path Integral Stochastic Optimal Control for Sampling Transition Paths »
Lars Holdijk · Yuanqi Du · Priyank Jaini · Ferry Hooft · Bernd Ensing · Max Welling -
2023 : Lie Point Symmetry and Physics Informed Networks »
Tara Akhound-Sadegh · Laurence Perreault-Levasseur · Johannes Brandstetter · Max Welling · Siamak Ravanbakhsh -
2023 Workshop: Structured Probabilistic Inference and Generative Modeling »
Dinghuai Zhang · Yuanqi Du · Chenlin Meng · Shawn Tan · Yingzhen Li · Max Welling · Yoshua Bengio -
2023 : Opening Remark »
Dinghuai Zhang · Yuanqi Du · Chenlin Meng · Shawn Tan · Yingzhen Li · Max Welling · Yoshua Bengio -
2023 Poster: Supervised Metric Learning to Rank for Retrieval via Contextual Similarity Optimization »
Christopher Liao · Theodoros Tsiligkaridis · Brian Kulis -
2023 Poster: Ske2Grid: Skeleton-to-Grid Representation Learning for Action Recognition »
Dongqi Cai · Yangyuxuan Kang · Anbang Yao · Yurong Chen -
2023 Poster: Neural Wave Machines: Learning Spatiotemporally Structured Representations with Locally Coupled Oscillatory Recurrent Neural Networks »
T. Anderson Keller · Max Welling -
2023 Poster: Latent Traversals in Generative Models as Potential Flows »
Yue Song · T. Anderson Keller · Nicu Sebe · Max Welling -
2023 Poster: Geometric Clifford Algebra Networks »
David Ruhe · Jayesh K. Gupta · Steven De Keninck · Max Welling · Johannes Brandstetter -
2022 Workshop: Machine Learning for Audio Synthesis »
Rachel Manzelli · Brian Kulis · Sadie Allen · Sander Dieleman · Yu Zhang -
2022 : Opening remarks »
Brian Kulis -
2022 Poster: Lie Point Symmetry Data Augmentation for Neural PDE Solvers »
Johannes Brandstetter · Max Welling · Daniel Worrall -
2022 Spotlight: Lie Point Symmetry Data Augmentation for Neural PDE Solvers »
Johannes Brandstetter · Max Welling · Daniel Worrall -
2022 Poster: Equivariant Diffusion for Molecule Generation in 3D »
Emiel Hoogeboom · Victor Garcia Satorras · Clément Vignac · Max Welling -
2022 Poster: ShiftAddNAS: Hardware-Inspired Search for More Accurate and Efficient Neural Networks »
Haoran You · Baopu Li · Shi Huihong · Yonggan Fu · Yingyan Lin -
2022 Poster: Faster Algorithms for Learning Convex Functions »
Ali Siahkamari · Durmus Alp Emre Acar · Christopher Liao · Kelly Geyer · Venkatesh Saligrama · Brian Kulis -
2022 Poster: DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks »
Yonggan Fu · Haichuan Yang · Jiayi Yuan · Meng Li · Cheng Wan · Raghuraman Krishnamoorthi · Vikas Chandra · Yingyan Lin -
2022 Spotlight: Faster Algorithms for Learning Convex Functions »
Ali Siahkamari · Durmus Alp Emre Acar · Christopher Liao · Kelly Geyer · Venkatesh Saligrama · Brian Kulis -
2022 Spotlight: DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks »
Yonggan Fu · Haichuan Yang · Jiayi Yuan · Meng Li · Cheng Wan · Raghuraman Krishnamoorthi · Vikas Chandra · Yingyan Lin -
2022 Oral: Equivariant Diffusion for Molecule Generation in 3D »
Emiel Hoogeboom · Victor Garcia Satorras · Clément Vignac · Max Welling -
2022 Spotlight: ShiftAddNAS: Hardware-Inspired Search for More Accurate and Efficient Neural Networks »
Haoran You · Baopu Li · Shi Huihong · Yonggan Fu · Yingyan Lin -
2021 Poster: Double-Win Quant: Aggressively Winning Robustness of Quantized Deep Neural Networks via Random Precision Training and Inference »
Yonggan Fu · Qixuan Yu · Meng Li · Vikas Chandra · Yingyan Lin -
2021 Test Of Time: Bayesian Learning via Stochastic Gradient Langevin Dynamics »
Yee Teh · Max Welling -
2021 Test Of Time: Test of Time Award »
Max Welling · Max Welling -
2021 Spotlight: Double-Win Quant: Aggressively Winning Robustness of Quantized Deep Neural Networks via Random Precision Training and Inference »
Yonggan Fu · Qixuan Yu · Meng Li · Vikas Chandra · Yingyan Lin -
2021 Poster: The Hintons in your Neural Network: a Quantum Field Theory View of Deep Learning »
Roberto Bondesan · Max Welling -
2021 Spotlight: The Hintons in your Neural Network: a Quantum Field Theory View of Deep Learning »
Roberto Bondesan · Max Welling -
2021 Poster: A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups »
Marc Finzi · Max Welling · Andrew Wilson -
2021 Poster: Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Accelerators »
Yonggan Fu · Yongan Zhang · Yang Zhang · David Cox · Yingyan Lin -
2021 Spotlight: Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Accelerators »
Yonggan Fu · Yongan Zhang · Yang Zhang · David Cox · Yingyan Lin -
2021 Oral: A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups »
Marc Finzi · Max Welling · Andrew Wilson -
2021 Poster: Federated Learning of User Verification Models Without Sharing Embeddings »
Hossein Hosseini · Hyunsin Park · Sungrack Yun · Christos Louizos · Joseph B Soriaga · Max Welling -
2021 Poster: E(n) Equivariant Graph Neural Networks »
Victor Garcia Satorras · Emiel Hoogeboom · Max Welling -
2021 Poster: Self Normalizing Flows »
T. Anderson Keller · Jorn Peters · Priyank Jaini · Emiel Hoogeboom · Patrick Forré · Max Welling -
2021 Spotlight: E(n) Equivariant Graph Neural Networks »
Victor Garcia Satorras · Emiel Hoogeboom · Max Welling -
2021 Spotlight: Federated Learning of User Verification Models Without Sharing Embeddings »
Hossein Hosseini · Hyunsin Park · Sungrack Yun · Christos Louizos · Joseph B Soriaga · Max Welling -
2021 Spotlight: Self Normalizing Flows »
T. Anderson Keller · Jorn Peters · Priyank Jaini · Emiel Hoogeboom · Patrick Forré · Max Welling -
2020 : Invited talk 1: Unifying VAEs and Flows »
Max Welling -
2020 Poster: Piecewise Linear Regression via a Difference of Convex Functions »
Ali Siahkamari · Aditya Gangrade · Brian Kulis · Venkatesh Saligrama -
2020 Poster: Involutive MCMC: a Unifying Framework »
Kirill Neklyudov · Max Welling · Evgenii Egorov · Dmitry Vetrov -
2020 Poster: Deep Divergence Learning »
Kubra Cilingir · Rachel Manzelli · Brian Kulis -
2020 Poster: AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks »
Yonggan Fu · Wuyang Chen · Haotao Wang · Haoran Li · Yingyan Lin · Zhangyang “Atlas” Wang -
2019 Workshop: Learning and Reasoning with Graph-Structured Representations »
Ethan Fetaya · Zhiting Hu · Thomas Kipf · Yujia Li · Xiaodan Liang · Renjie Liao · Raquel Urtasun · Hao Wang · Max Welling · Eric Xing · Richard Zemel -
2019 : Poster discussion »
Roman Novak · Maxime Gabella · Frederic Dreyer · Siavash Golkar · Anh Tong · Irina Higgins · Mirco Milletari · Joe Antognini · Sebastian Goldt · Adín Ramírez Rivera · Roberto Bondesan · Ryo Karakida · Remi Tachet des Combes · Michael Mahoney · Nicholas Walker · Stanislav Fort · Samuel Smith · Rohan Ghosh · Aristide Baratin · Diego Granziol · Stephen Roberts · Dmitry Vetrov · Andrew Wilson · César Laurent · Valentin Thomas · Simon Lacoste-Julien · Dar Gilboa · Daniel Soudry · Anupam Gupta · Anirudh Goyal · Yoshua Bengio · Erich Elsen · Soham De · Stanislaw Jastrzebski · Charles H Martin · Samira Shabanian · Aaron Courville · Shorato Akaho · Lenka Zdeborova · Ethan Dyer · Maurice Weiler · Pim de Haan · Taco Cohen · Max Welling · Ping Luo · zhanglin peng · Nasim Rahaman · Loic Matthey · Danilo J. Rezende · Jaesik Choi · Kyle Cranmer · Lechao Xiao · Jaehoon Lee · Yasaman Bahri · Jeffrey Pennington · Greg Yang · Jiri Hron · Jascha Sohl-Dickstein · Guy Gur-Ari -
2019 : Panel Discussion (moderator: Tom Dietterich) »
Max Welling · Kilian Weinberger · Terrance Boult · Dawn Song · Thomas Dietterich -
2019 : Mixed Precision Training & Inference »
Jonathan Dekhtiar -
2019 : Keynote by Max Welling: A Nonparametric Bayesian Approach to Deep Learning (without GPs) »
Max Welling -
2019 Workshop: Theoretical Physics for Deep Learning »
Jaehoon Lee · Jeffrey Pennington · Yasaman Bahri · Max Welling · Surya Ganguli · Joan Bruna -
2019 : Opening Remarks »
Jaehoon Lee · Jeffrey Pennington · Yasaman Bahri · Max Welling · Surya Ganguli · Joan Bruna -
2019 Poster: Efficient On-Device Models using Neural Projections »
Sujith Ravi -
2019 Poster: Stochastic Beams and Where To Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement »
Wouter Kool · Herke van Hoof · Max Welling -
2019 Oral: Efficient On-Device Models using Neural Projections »
Sujith Ravi -
2019 Oral: Stochastic Beams and Where To Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement »
Wouter Kool · Herke van Hoof · Max Welling -
2019 Poster: Emerging Convolutions for Generative Normalizing Flows »
Emiel Hoogeboom · Rianne Van den Berg · Max Welling -
2019 Oral: Emerging Convolutions for Generative Normalizing Flows »
Emiel Hoogeboom · Rianne Van den Berg · Max Welling -
2019 Poster: Gauge Equivariant Convolutional Networks and the Icosahedral CNN »
Taco Cohen · Maurice Weiler · Berkay Kicanaoglu · Max Welling -
2019 Oral: Gauge Equivariant Convolutional Networks and the Icosahedral CNN »
Taco Cohen · Maurice Weiler · Berkay Kicanaoglu · Max Welling -
2018 Poster: Attention-based Deep Multiple Instance Learning »
Maximilian Ilse · Jakub Tomczak · Max Welling -
2018 Poster: Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions »
Junru Wu · Yue Wang · Zhenyu Wu · Zhangyang Wang · Ashok Veeraraghavan · Yingyan Lin -
2018 Oral: Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions »
Junru Wu · Yue Wang · Zhenyu Wu · Zhangyang Wang · Ashok Veeraraghavan · Yingyan Lin -
2018 Oral: Attention-based Deep Multiple Instance Learning »
Maximilian Ilse · Jakub Tomczak · Max Welling -
2018 Invited Talk: Intelligence per Kilowatthour »
Max Welling -
2018 Poster: Neural Relational Inference for Interacting Systems »
Thomas Kipf · Ethan Fetaya · Kuan-Chieh Wang · Max Welling · Richard Zemel -
2018 Poster: BOCK : Bayesian Optimization with Cylindrical Kernels »
ChangYong Oh · Efstratios Gavves · Max Welling -
2018 Oral: Neural Relational Inference for Interacting Systems »
Thomas Kipf · Ethan Fetaya · Kuan-Chieh Wang · Max Welling · Richard Zemel -
2018 Oral: BOCK : Bayesian Optimization with Cylindrical Kernels »
ChangYong Oh · Efstratios Gavves · Max Welling -
2017 Poster: Multiplicative Normalizing Flows for Variational Bayesian Neural Networks »
Christos Louizos · Max Welling -
2017 Talk: Multiplicative Normalizing Flows for Variational Bayesian Neural Networks »
Christos Louizos · Max Welling