Timezone: »
Invertible neural networks have been a significant thread of research in the ICML community for several years. Such transformations can offer a range of unique benefits:
(1) They preserve information, allowing perfect reconstruction (up to numerical limits) and obviating the need to store hidden activations in memory for backpropagation.
(2) They are often designed to track the changes in probability density that applying the transformation induces (as in normalizing flows).
(3) Like autoregressive models, normalizing flows can be powerful generative models which allow exact likelihood computations; with the right architecture, they can also allow for much cheaper sampling than autoregressive models.
While many researchers are aware of these topics and intrigued by several high-profile papers, few are familiar enough with the technical details to easily follow new developments and contribute. Many may also be unaware of the wide range of applications of invertible neural networks, beyond generative modelling and variational inference.
Sat 9:30 a.m. - 10:30 a.m.
|
Tutorial on normalizing flows
(Tutorial)
Video »
|
Eric Jang |
Sat 10:30 a.m. - 10:50 a.m.
|
Poster Spotlights
(Talk)
|
|
Sat 10:50 a.m. - 11:30 a.m.
|
poster session I
(Poster session)
|
Nicholas Rhinehart, Yunhao Tang, Vinay Prabhu, Dian Ang Yap, Alexander Wang, Marc Finzi, Manoj Kumar, You Lu, Abhishek Kumar, Qi Lei, Michael Przystupa, Nicola De Cao, Polina Kirichenko, Pavel Izmailov, Andrew Wilson, Jakob Kruse, Diego Mesquita, Mario Lezcano Casado, Thomas Müller, Keir Simmons, Andrei Atanov
|
Sat 11:30 a.m. - 11:50 a.m.
|
Building a tractable generator network
(Talk)
Video »
|
|
Sat 11:50 a.m. - 12:10 p.m.
|
Glow: Generative Flow with Invertible 1x1 Convolutions
(Talk)
Video »
|
Prafulla Dhariwal |
Sat 12:10 p.m. - 12:30 p.m.
|
Contributed talk
(Talk)
Video »
|
|
Sat 2:00 p.m. - 2:20 p.m.
|
Householder meets Sylvester: Normalizing flows for variational inference
(Talk)
Video »
|
|
Sat 2:20 p.m. - 2:40 p.m.
|
Neural Ordinary Differential Equations for Continuous Normalizing Flows
(Talk)
Video »
|
|
Sat 2:40 p.m. - 3:00 p.m.
|
Contributed talk
(Talk)
Video »
|
|
Sat 3:00 p.m. - 4:00 p.m.
|
poster session II
(Poster session)
|
|
Sat 4:00 p.m. - 4:20 p.m.
|
The Bijector API: An Invertible Function Library for TensorFlow
(Talk)
Video »
|
|
Sat 4:20 p.m. - 4:40 p.m.
|
Invertible Neural Networks for Understanding and Controlling Learned Representations
(Talk)
Video »
|
|
Sat 4:40 p.m. - 5:00 p.m.
|
Contributed talk
(Talk)
Video »
|
|
Sat 5:00 p.m. - 6:00 p.m.
|
Panel Session
(Panel)
Video »
|
Author Information
Chin-Wei Huang (MILA)
David Krueger (Universit? de Montr?al)
Rianne Van den Berg (University of Amsterdam)
George Papamakarios (University of Edinburgh)
Aidan Gomez (University of Oxford)
Chris Cremer (University of Toronto)
Aaron Courville (Université de Montréal)
Ricky T. Q. Chen (U of Toronto)
Danilo J. Rezende (DeepMind)

Danilo is a Senior Staff Research Scientist at Google DeepMind, where he works on probabilistic machine reasoning and learning algorithms. He has a BA in Physics and MSc in Theoretical Physics from Ecole Polytechnique (Palaiseau – France) and from the Institute of Theoretical Physics (SP – Brazil) and a Ph.D. in Computational Neuroscience at Ecole Polytechnique Federale de Lausanne, EPFL (Lausanne – Switzerland). His research focuses on scalable inference methods, generative models of complex data (such as images and video), applied probability, causal reasoning and unsupervised learning for decision-making.
More from the Same Authors
-
2020 Workshop: INNF+: Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models »
Chin-Wei Huang · David Krueger · Rianne Van den Berg · George Papamakarios · Chris Cremer · Tian Qi Chen · Danilo J. Rezende -
2020 Poster: AR-DAE: Towards Unbiased Neural Entropy Gradient Estimation »
Jae Hyun Lim · Aaron Courville · Christopher Pal · Chin-Wei Huang -
2020 Poster: Countering Language Drift with Seeded Iterated Learning »
Yuchen Lu · Soumye Singhal · Florian Strub · Aaron Courville · Olivier Pietquin -
2020 Poster: Normalizing Flows on Tori and Spheres »
Danilo J. Rezende · George Papamakarios · Sebastien Racaniere · Michael Albergo · Gurtej Kanwar · Phiala Shanahan · Kyle Cranmer -
2020 Tutorial: Representation Learning Without Labels »
S. M. Ali Eslami · Irina Higgins · Danilo J. Rezende -
2019 Poster: Hierarchical Importance Weighted Autoencoders »
Chin-Wei Huang · Kris Sankaran · Eeshan Dhekane · Alexandre Lacoste · Aaron Courville -
2019 Oral: Hierarchical Importance Weighted Autoencoders »
Chin-Wei Huang · Kris Sankaran · Eeshan Dhekane · Alexandre Lacoste · Aaron Courville -
2019 Poster: Emerging Convolutions for Generative Normalizing Flows »
Emiel Hoogeboom · Rianne Van den Berg · Max Welling -
2019 Poster: Invertible Residual Networks »
Jens Behrmann · Will Grathwohl · Tian Qi Chen · David Duvenaud · Joern-Henrik Jacobsen -
2019 Oral: Invertible Residual Networks »
Jens Behrmann · Will Grathwohl · Tian Qi Chen · David Duvenaud · Joern-Henrik Jacobsen -
2019 Oral: Emerging Convolutions for Generative Normalizing Flows »
Emiel Hoogeboom · Rianne Van den Berg · Max Welling -
2018 Poster: Neural Autoregressive Flows »
Chin-Wei Huang · David Krueger · Alexandre Lacoste · Aaron Courville -
2018 Oral: Neural Autoregressive Flows »
Chin-Wei Huang · David Krueger · Alexandre Lacoste · Aaron Courville -
2018 Poster: Generative Temporal Models with Spatial Memory for Partially Observed Environments »
Marco Fraccaro · Danilo J. Rezende · Yori Zwols · Alexander Pritzel · S. M. Ali Eslami · Fabio Viola -
2018 Poster: Conditional Neural Processes »
Marta Garnelo · Dan Rosenbaum · Chris Maddison · Tiago Ramalho · David Saxton · Murray Shanahan · Yee Teh · Danilo J. Rezende · S. M. Ali Eslami -
2018 Poster: Inference Suboptimality in Variational Autoencoders »
Chris Cremer · Xuechen Li · David Duvenaud -
2018 Oral: Inference Suboptimality in Variational Autoencoders »
Chris Cremer · Xuechen Li · David Duvenaud -
2018 Oral: Generative Temporal Models with Spatial Memory for Partially Observed Environments »
Marco Fraccaro · Danilo J. Rezende · Yori Zwols · Alexander Pritzel · S. M. Ali Eslami · Fabio Viola -
2018 Oral: Conditional Neural Processes »
Marta Garnelo · Dan Rosenbaum · Chris Maddison · Tiago Ramalho · David Saxton · Murray Shanahan · Yee Teh · Danilo J. Rezende · S. M. Ali Eslami