Timezone: »

Joint Workshop on On-Device Machine Learning & Compact Deep Neural Network Representations (ODML-CDNNR)
Sujith Ravi · Zornitsa Kozareva · Lixin Fan · Max Welling · Yurong Chen · Werner Bailer · Brian Kulis · Haoji Hu · Jonathan Dekhtiar · Yingyan Lin · Diana Marculescu

Fri Jun 14 08:30 AM -- 06:00 PM (PDT) @ 203
Event URL: https://sites.google.com/corp/view/icml2019-on-device-compact-dnn »

This joint workshop aims to bring together researchers, educators, practitioners who are interested in techniques as well as applications of on-device machine learning and compact, efficient neural network representations. One aim of the workshop discussion is to establish close connection between researchers in the machine learning community and engineers in industry, and to benefit both academic researchers as well as industrial practitioners. The other aim is the evaluation and comparability of resource-efficient machine learning methods and compact and efficient network representations, and their relation to particular target platforms (some of which may be highly optimized for neural network inference). The research community has still to develop established evaluation procedures and metrics.

The workshop also aims at reproducibility and comparability of methods for compact and efficient neural network representations, and on-device machine learning. Contributors are thus encouraged to make their code available. The workshop organizers plan to make some example tasks and datasets available, and invite contributors to use them for testing their work. In order to provide comparable performance evaluation conditions, the use of a common platform (such as Google Colab) is intended.

Fri 8:30 a.m. - 8:40 a.m.
Welcome and Introduction (Talk)
Fri 8:40 a.m. - 9:10 a.m.
Hardware Efficiency Aware Neural Architecture Search and Compression (Invited talk) [ Video
Song Han
Fri 9:10 a.m. - 9:40 a.m.
Structured matrices for efficient deep learning (Invited talk) [ Video
Sanjiv Kumar
Fri 9:40 a.m. - 10:00 a.m.
[ Video

Simon Wiedemann, Heiner Kirchhoffer, Stefan Matlage, Paul Haase, Arturo Marban Gonzalez, Talmaj Marinc, Heiko Schwarz, Detlev Marpe, Thomas Wiegand, Ahmed Osman and Wojciech Samek


Simon Wiedemann
Fri 10:00 a.m. - 10:30 a.m.

2min presentations of the posters presentations during lunch break

Fri 10:30 a.m. - 11:00 a.m.
Coffee Break AM (Break)
Fri 11:00 a.m. - 11:30 a.m.
[ Video

The co-design of algorithm and hardware has become an increasingly important approach for addressing the computational complexity of Deep Neural Networks (DNNs). There are several open problems and challenges in the co-design process and application; for instance, what metrics should be used to drive the algorithm design, how to automate the process in a simple way, how to extend these approaches to tasks beyond image classification, and how to design flexible hardware to support these different approaches. In this talk, we highlight recent and ongoing work that aim to address these challenges, namely energy-aware pruning and NetAdapt that automatically incorporate direct metrics such as latency and energy into the training and design of the DNN; FastDepth that extends the co-design approaches to a depth estimation task; and a flexible hardware accelerator called Eyeriss v2 that is computationally efficient across a wide range of diverse DNNs. BIO: Vivienne Sze is an Associate Professor at MIT in the Electrical Engineering and Computer Science Department. Her research interests include energy-aware signal processing algorithms, and low-power circuit and system design for portable multimedia applications, including computer vision, deep learning, autonomous navigation, and video process/coding. Prior to joining MIT, she was a Member of Technical Staff in the R&D Center at TI, where she designed low-power algorithms and architectures for video coding. She also represented TI in the JCT-VC committee of ITU-T and ISO/IEC standards body during the development of High Efficiency Video Coding (HEVC), which received a Primetime Engineering Emmy Award. She is a co-editor of the book entitled “High Efficiency Video Coding (HEVC): Algorithms and Architectures” (Springer, 2014).
Prof. Sze received the B.A.Sc. degree from the University of Toronto in 2004, and the S.M. and Ph.D. degree from MIT in 2006 and 2010, respectively. In 2011, she received the Jin-Au Kong Outstanding Doctoral Thesis Prize in Electrical Engineering at MIT. She is a recipient of the 2019 Edgerton Faculty Award, the 2018 Facebook Faculty Award, the 2018 & 2017 Qualcomm Faculty Award, the 2018 & 2016 Google Faculty Research Award, the 2016 AFOSR Young Investigator Research Program (YIP) Award, the 2016 3M Non-Tenured Faculty Award, the 2014 DARPA Young Faculty Award, the 2007 DAC/ISSCC Student Design Contest Award, and a co-recipient of the 2017 CICC Outstanding Invited Paper Award, the 2016 IEEE Micro Top Picks Award and the 2008 A-SSCC Outstanding Design Award. For more information about research in the Energy-Efficient Multimedia Systems Group at MIT visit: http://www.rle.mit.edu/eems/

Vivienne Sze
Fri 11:30 a.m. - 11:50 a.m.
[ Video

Kartikeya Bhardwaj, Naveen Suda and Radu Marculescu


Kartikeya Bhardwaj
Fri 11:50 a.m. - 12:10 p.m.
[ Video

Trevor Gale, Erich Elsen and Sara Hooker

https://arxiv.org/abs/1902.09574 (to be updated)

Trevor Gale
Fri 12:10 p.m. - 12:40 p.m.
Lunch break (Break)
Fri 12:40 p.m. - 2:00 p.m.

Xiaofan Zhang, Hao Cong, Yuhong Li, Yao Chen, Jinjun Xiong, Wen-Mei Hwu and Deming Chen. A Bi-Directional Co-Design Approach to Enable Deep Learning on IoT Devices https://arxiv.org/abs/1905.08369

Kushal Datta, Aishwarya Bhandare, Deepthi Karkada, Vamsi Sripathi, Sun Choi, Vikram Saletore and Vivek Menon. Efficient 8-Bit Quantization of Transformer Neural Machine Language Translation Model

Bin Yang, Lin Yang, Xiaochun Li, Wenhan Zhang, Hua Zhou, Yequn Zhang, Yongxiong Ren and Yinbo Shi. 2-bit Model Compression of Deep Convolutional Neural Network on ASIC Engine for image retrieval https://arxiv.org/abs/1905.03362

Zhong Qiu Lin, Brendan Chwyl and Alexander Wong. EdgeSegNet: A Compact Network for Semantic Segmentation https://arxiv.org/abs/1905.04222

Sheng Lin, Xiaolong Ma, Shaokai Ye, Geng Yuan, Kaisheng Ma and Yanzhi Wang. Toward Extremely Low Bit and Lossless Accuracy in DNNs with Progressive ADMM https://arxiv.org/abs/1905.00789

Chengcheng Li, Zi Wang, Dali Wang, Xiangyang Wang and Hairong Qi. Investigating Channel Pruning through Structural Redundancy Reduction - A Statistical Study https://arxiv.org/abs/1905.06498

Wei Niu, Yanzhi Wang and Bin Ren. CADNN: Ultra Fast Execution of DNNs on Mobile Devices with Advanced Model Compression and Architecture-Aware Optimization https://arxiv.org/abs/1905.00571

Jonathan Ephrath, Lars Ruthotto, Eldad Haber and Eran Treister. LeanResNet: A Low-cost yet Effective Convolutional Residual Networks https://arxiv.org/abs/1904.06952

Dushyant Mehta, Kwang In Kim and Christian Theobalt. Implicit Filter Sparsification In Convolutional Neural Networks https://arxiv.org/abs/1905.04967

Cong Hao, Zhongqiu Lin, Chengcheng Li, Lars Ruthotto, Bin Yang, Deepthi Karkada
Fri 2:00 p.m. - 2:30 p.m.
[ Video

Kailash Gopalakrishnan

Kailash Gopalakrishnan
Fri 2:30 p.m. - 3:00 p.m.
Mixed Precision Training & Inference (Invited talk) [ Video
Jonathan Dekhtiar
Fri 3:00 p.m. - 3:30 p.m.
Coffee Break PM (Break)
Fri 3:30 p.m. - 3:50 p.m.
[ Video

Mohamadali Torkamani, Phillip Wallis, Shiv Shankar and Amirmohammad Rooshenas


Fri 3:50 p.m. - 4:10 p.m.

Yushu Feng, Huan Wang and Haoji Hu


Fri 4:10 p.m. - 4:30 p.m.
[ Video

Dimitrios Stamoulis, Ruizhou Ding, Di Wang, Dimitrios Lymberopoulos, Bodhi Priyantha, Jie Liu and Diana Marculescu


Dimitrios Stamoulis
Fri 4:30 p.m. - 5:30 p.m.
Panel discussion (Discussion) [ Video
Fri 5:30 p.m. - 5:45 p.m.
Wrap-up and Closing (Talk)

Author Information

Sujith Ravi (Google Research)
Zornitsa Kozareva (Google)
Lixin Fan (JD.com)
Max Welling (University of Amsterdam & Qualcomm)
Yurong Chen (Intel)
Brian Kulis (Boston University)
Haoji Hu (Zhejiang University)
Jonathan Dekhtiar (NVIDIA)
Yingyan Lin (Rice University)
Diana Marculescu (Carnegie Mellon University)

More from the Same Authors