Timezone: »
Invertible neural networks have been a significant thread of research in the ICML community for several years. Such transformations can offer a range of unique benefits:
(1) They preserve information, allowing perfect reconstruction (up to numerical limits) and obviating the need to store hidden activations in memory for backpropagation.
(2) They are often designed to track the changes in probability density that applying the transformation induces (as in normalizing flows).
(3) Like autoregressive models, normalizing flows can be powerful generative models which allow exact likelihood computations; with the right architecture, they can also allow for much cheaper sampling than autoregressive models.
While many researchers are aware of these topics and intrigued by several high-profile papers, few are familiar enough with the technical details to easily follow new developments and contribute. Many may also be unaware of the wide range of applications of invertible neural networks, beyond generative modelling and variational inference.
Sat 9:30 a.m. - 10:30 a.m.
|
Tutorial on normalizing flows
(
Tutorial
)
|
Eric Jang 🔗 |
Sat 10:30 a.m. - 10:50 a.m.
|
Poster Spotlights
(
Talk
)
|
🔗 |
Sat 10:50 a.m. - 11:30 a.m.
|
poster session I
(
Poster session
)
|
Nicholas Rhinehart · Yunhao Tang · Vinay Prabhu · Dian Ang Yap · Alexander Wang · Marc Finzi · Manoj Kumar · You Lu · Abhishek Kumar · Qi Lei · Michael Przystupa · Nicola De Cao · Polina Kirichenko · Pavel Izmailov · Andrew Wilson · Jakob Kruse · Diego Mesquita · Mario Lezcano Casado · Thomas Müller · Keir Simmons · Andrei Atanov
|
Sat 11:30 a.m. - 11:50 a.m.
|
Building a tractable generator network
(
Talk
)
|
🔗 |
Sat 11:50 a.m. - 12:10 p.m.
|
Glow: Generative Flow with Invertible 1x1 Convolutions
(
Talk
)
|
Prafulla Dhariwal 🔗 |
Sat 12:10 p.m. - 12:30 p.m.
|
Contributed talk
(
Talk
)
|
🔗 |
Sat 2:00 p.m. - 2:20 p.m.
|
Householder meets Sylvester: Normalizing flows for variational inference
(
Talk
)
|
🔗 |
Sat 2:20 p.m. - 2:40 p.m.
|
Neural Ordinary Differential Equations for Continuous Normalizing Flows
(
Talk
)
|
🔗 |
Sat 2:40 p.m. - 3:00 p.m.
|
Contributed talk
(
Talk
)
|
🔗 |
Sat 3:00 p.m. - 4:00 p.m.
|
poster session II
(
Poster session
)
|
🔗 |
Sat 4:00 p.m. - 4:20 p.m.
|
The Bijector API: An Invertible Function Library for TensorFlow
(
Talk
)
|
🔗 |
Sat 4:20 p.m. - 4:40 p.m.
|
Invertible Neural Networks for Understanding and Controlling Learned Representations
(
Talk
)
|
🔗 |
Sat 4:40 p.m. - 5:00 p.m.
|
Contributed talk
(
Talk
)
|
🔗 |
Sat 5:00 p.m. - 6:00 p.m.
|
Panel Session
(
Panel
)
|
🔗 |
Author Information
Chin-Wei Huang (MILA)
David Krueger (Universit? de Montr?al)
Rianne Van den Berg (University of Amsterdam)
George Papamakarios (University of Edinburgh)
Aidan Gomez (University of Oxford)
Chris Cremer (University of Toronto)
Aaron Courville (Université de Montréal)
Ricky T. Q. Chen (U of Toronto)
Danilo J. Rezende (DeepMind)

Danilo is a Senior Staff Research Scientist at Google DeepMind, where he works on probabilistic machine reasoning and learning algorithms. He has a BA in Physics and MSc in Theoretical Physics from Ecole Polytechnique (Palaiseau – France) and from the Institute of Theoretical Physics (SP – Brazil) and a Ph.D. in Computational Neuroscience at Ecole Polytechnique Federale de Lausanne, EPFL (Lausanne – Switzerland). His research focuses on scalable inference methods, generative models of complex data (such as images and video), applied probability, causal reasoning and unsupervised learning for decision-making.
More from the Same Authors
-
2021 : A Variational Perspective on Diffusion-Based Generative Models and Score Matching »
Chin-Wei Huang -
2022 : Learning to induce causal structure »
Rosemary Nan Ke · Silvia Chiappa · Jane Wang · Jorg Bornschein · Anirudh Goyal · Melanie Rey · Matthew Botvinick · Theophane Weber · Michael Mozer · Danilo J. Rezende -
2022 Poster: Building Robust Ensembles via Margin Boosting »
Dinghuai Zhang · Hongyang Zhang · Aaron Courville · Yoshua Bengio · Pradeep Ravikumar · Arun Sai Suggala -
2022 Spotlight: Building Robust Ensembles via Margin Boosting »
Dinghuai Zhang · Hongyang Zhang · Aaron Courville · Yoshua Bengio · Pradeep Ravikumar · Arun Sai Suggala -
2022 Poster: From data to functa: Your data point is a function and you can treat it like one »
Emilien Dupont · Hyunjik Kim · S. M. Ali Eslami · Danilo J. Rezende · Dan Rosenbaum -
2022 Poster: Goal Misgeneralization in Deep Reinforcement Learning »
Lauro Langosco di Langosco · Jack Koch · Lee Sharkey · Jacob Pfau · David Krueger -
2022 Spotlight: Goal Misgeneralization in Deep Reinforcement Learning »
Lauro Langosco di Langosco · Jack Koch · Lee Sharkey · Jacob Pfau · David Krueger -
2022 Spotlight: From data to functa: Your data point is a function and you can treat it like one »
Emilien Dupont · Hyunjik Kim · S. M. Ali Eslami · Danilo J. Rezende · Dan Rosenbaum -
2022 Poster: Generative Flow Networks for Discrete Probabilistic Modeling »
Dinghuai Zhang · Nikolay Malkin · Zhen Liu · Alexandra Volokhova · Aaron Courville · Yoshua Bengio -
2022 Poster: Continual Repeated Annealed Flow Transport Monte Carlo »
Alexander Matthews · Michael Arbel · Danilo J. Rezende · Arnaud Doucet -
2022 Poster: The Primacy Bias in Deep Reinforcement Learning »
Evgenii Nikishin · Max Schwarzer · Pierluca D'Oro · Pierre-Luc Bacon · Aaron Courville -
2022 Spotlight: Generative Flow Networks for Discrete Probabilistic Modeling »
Dinghuai Zhang · Nikolay Malkin · Zhen Liu · Alexandra Volokhova · Aaron Courville · Yoshua Bengio -
2022 Spotlight: Continual Repeated Annealed Flow Transport Monte Carlo »
Alexander Matthews · Michael Arbel · Danilo J. Rezende · Arnaud Doucet -
2022 Spotlight: The Primacy Bias in Deep Reinforcement Learning »
Evgenii Nikishin · Max Schwarzer · Pierluca D'Oro · Pierre-Luc Bacon · Aaron Courville -
2021 Workshop: INNF+: Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models »
Chin-Wei Huang · David Krueger · Rianne Van den Berg · George Papamakarios · Ricky T. Q. Chen · Danilo J. Rezende -
2021 Poster: Can Subnetwork Structure Be the Key to Out-of-Distribution Generalization? »
Dinghuai Zhang · Kartik Ahuja · Yilun Xu · Yisen Wang · Aaron Courville -
2021 Oral: Can Subnetwork Structure Be the Key to Out-of-Distribution Generalization? »
Dinghuai Zhang · Kartik Ahuja · Yilun Xu · Yisen Wang · Aaron Courville -
2021 Poster: Continuous Coordination As a Realistic Scenario for Lifelong Learning »
Hadi Nekoei · Akilesh Badrinaaraayanan · Aaron Courville · Sarath Chandar -
2021 Spotlight: Continuous Coordination As a Realistic Scenario for Lifelong Learning »
Hadi Nekoei · Akilesh Badrinaaraayanan · Aaron Courville · Sarath Chandar -
2021 Poster: Out-of-Distribution Generalization via Risk Extrapolation (REx) »
David Krueger · Ethan Caballero · Joern-Henrik Jacobsen · Amy Zhang · Jonathan Binas · Dinghuai Zhang · Remi Le Priol · Aaron Courville -
2021 Oral: Out-of-Distribution Generalization via Risk Extrapolation (REx) »
David Krueger · Ethan Caballero · Joern-Henrik Jacobsen · Amy Zhang · Jonathan Binas · Dinghuai Zhang · Remi Le Priol · Aaron Courville -
2021 Oral: NeRF-VAE: A Geometry Aware 3D Scene Generative Model »
Adam Kosiorek · Heiko Strathmann · Daniel Zoran · Pol Moreno · Rosalia Schneider · Sona Mokra · Danilo J. Rezende -
2021 Poster: NeRF-VAE: A Geometry Aware 3D Scene Generative Model »
Adam Kosiorek · Heiko Strathmann · Daniel Zoran · Pol Moreno · Rosalia Schneider · Sona Mokra · Danilo J. Rezende -
2021 Poster: "Hey, that's not an ODE": Faster ODE Adjoints via Seminorms »
Patrick Kidger · Ricky T. Q. Chen · Terry Lyons -
2021 Spotlight: "Hey, that's not an ODE": Faster ODE Adjoints via Seminorms »
Patrick Kidger · Ricky T. Q. Chen · Terry Lyons -
2020 Workshop: INNF+: Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models »
Chin-Wei Huang · David Krueger · Rianne Van den Berg · George Papamakarios · Chris Cremer · Ricky T. Q. Chen · Danilo J. Rezende -
2020 Poster: AR-DAE: Towards Unbiased Neural Entropy Gradient Estimation »
Jae Hyun Lim · Aaron Courville · Christopher Pal · Chin-Wei Huang -
2020 Poster: Countering Language Drift with Seeded Iterated Learning »
Yuchen Lu · Soumye Singhal · Florian Strub · Aaron Courville · Olivier Pietquin -
2020 Poster: Normalizing Flows on Tori and Spheres »
Danilo J. Rezende · George Papamakarios · Sebastien Racaniere · Michael Albergo · Gurtej Kanwar · Phiala Shanahan · Kyle Cranmer -
2020 Tutorial: Representation Learning Without Labels »
S. M. Ali Eslami · Irina Higgins · Danilo J. Rezende -
2019 : Poster discussion »
Roman Novak · Maxime Gabella · Frederic Dreyer · Siavash Golkar · Anh Tong · Irina Higgins · Mirco Milletari · Joe Antognini · Sebastian Goldt · Adín Ramírez Rivera · Roberto Bondesan · Ryo Karakida · Remi Tachet des Combes · Michael Mahoney · Nicholas Walker · Stanislav Fort · Samuel Smith · Rohan Ghosh · Aristide Baratin · Diego Granziol · Stephen Roberts · Dmitry Vetrov · Andrew Wilson · César Laurent · Valentin Thomas · Simon Lacoste-Julien · Dar Gilboa · Daniel Soudry · Anupam Gupta · Anirudh Goyal · Yoshua Bengio · Erich Elsen · Soham De · Stanislaw Jastrzebski · Charles H Martin · Samira Shabanian · Aaron Courville · Shorato Akaho · Lenka Zdeborova · Ethan Dyer · Maurice Weiler · Pim de Haan · Taco Cohen · Max Welling · Ping Luo · zhanglin peng · Nasim Rahaman · Loic Matthey · Danilo J. Rezende · Jaesik Choi · Kyle Cranmer · Lechao Xiao · Jaehoon Lee · Yasaman Bahri · Jeffrey Pennington · Greg Yang · Jiri Hron · Jascha Sohl-Dickstein · Guy Gur-Ari -
2019 Poster: Hierarchical Importance Weighted Autoencoders »
Chin-Wei Huang · Kris Sankaran · Eeshan Dhekane · Alexandre Lacoste · Aaron Courville -
2019 Oral: Hierarchical Importance Weighted Autoencoders »
Chin-Wei Huang · Kris Sankaran · Eeshan Dhekane · Alexandre Lacoste · Aaron Courville -
2019 Poster: Emerging Convolutions for Generative Normalizing Flows »
Emiel Hoogeboom · Rianne Van den Berg · Max Welling -
2019 Poster: Invertible Residual Networks »
Jens Behrmann · Will Grathwohl · Ricky T. Q. Chen · David Duvenaud · Joern-Henrik Jacobsen -
2019 Oral: Invertible Residual Networks »
Jens Behrmann · Will Grathwohl · Ricky T. Q. Chen · David Duvenaud · Joern-Henrik Jacobsen -
2019 Oral: Emerging Convolutions for Generative Normalizing Flows »
Emiel Hoogeboom · Rianne Van den Berg · Max Welling -
2018 Poster: Neural Autoregressive Flows »
Chin-Wei Huang · David Krueger · Alexandre Lacoste · Aaron Courville -
2018 Oral: Neural Autoregressive Flows »
Chin-Wei Huang · David Krueger · Alexandre Lacoste · Aaron Courville -
2018 Poster: Generative Temporal Models with Spatial Memory for Partially Observed Environments »
Marco Fraccaro · Danilo J. Rezende · Yori Zwols · Alexander Pritzel · S. M. Ali Eslami · Fabio Viola -
2018 Poster: Conditional Neural Processes »
Marta Garnelo · Dan Rosenbaum · Chris Maddison · Tiago Ramalho · David Saxton · Murray Shanahan · Yee Teh · Danilo J. Rezende · S. M. Ali Eslami -
2018 Poster: Inference Suboptimality in Variational Autoencoders »
Chris Cremer · Xuechen Li · David Duvenaud -
2018 Oral: Inference Suboptimality in Variational Autoencoders »
Chris Cremer · Xuechen Li · David Duvenaud -
2018 Oral: Generative Temporal Models with Spatial Memory for Partially Observed Environments »
Marco Fraccaro · Danilo J. Rezende · Yori Zwols · Alexander Pritzel · S. M. Ali Eslami · Fabio Viola -
2018 Oral: Conditional Neural Processes »
Marta Garnelo · Dan Rosenbaum · Chris Maddison · Tiago Ramalho · David Saxton · Murray Shanahan · Yee Teh · Danilo J. Rezende · S. M. Ali Eslami