Timezone: »
We show that normalising flows become pathological when used to model targets whose supports have complicated topologies. In this scenario, we prove that a flow must become arbitrarily numerically noninvertible in order to approximate the target closely. This result has implications for all flow-based models, and especially residual flows (ResFlows), which explicitly control the Lipschitz constant of the bijection used. To address this, we propose continuously indexed flows (CIFs), which replace the single bijection used by normalising flows with a continuously indexed family of bijections, and which can intuitively "clean up" mass that would otherwise be misplaced by a single bijection. We show theoretically that CIFs are not subject to the same topological limitations as normalising flows, and obtain better empirical performance on a variety of models and benchmarks.
Author Information
Rob Cornish (University of Oxford)
Anthony Caterini (University of Oxford)
George Deligiannidis (Oxford)
Arnaud Doucet (Oxford University)
More from the Same Authors
-
2021 : Rectangular Flows for Manifold Learning »
Anthony Caterini -
2022 : Riemannian Diffusion Schr\"odinger Bridge »
James Thornton · Valentin De Bortoli · Michael Hutchinson · Emile Mathieu · Yee Whye Teh · Arnaud Doucet -
2021 Poster: Monte Carlo Variational Auto-Encoders »
Achille Thin · Nikita Kotelevskii · Arnaud Doucet · Alain Durmus · Eric Moulines · Maxim Panov -
2021 Spotlight: Monte Carlo Variational Auto-Encoders »
Achille Thin · Nikita Kotelevskii · Arnaud Doucet · Alain Durmus · Eric Moulines · Maxim Panov -
2021 Poster: Differentiable Particle Filtering via Entropy-Regularized Optimal Transport »
Adrien Corenflos · James Thornton · George Deligiannidis · Arnaud Doucet -
2021 Oral: Differentiable Particle Filtering via Entropy-Regularized Optimal Transport »
Adrien Corenflos · James Thornton · George Deligiannidis · Arnaud Doucet -
2021 Poster: Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding »
Yangjun Ruan · Karen Ullrich · Daniel Severo · James Townsend · Ashish Khisti · Arnaud Doucet · Alireza Makhzani · Chris Maddison -
2021 Oral: Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding »
Yangjun Ruan · Karen Ullrich · Daniel Severo · James Townsend · Ashish Khisti · Arnaud Doucet · Alireza Makhzani · Chris Maddison -
2019 : Spotlight »
Tyler Scott · Kiran Koshy · Jonathan Aigrain · Rene Bidart · Priyadarshini Panda · Dian Ang Yap · Yaniv Yacoby · Raphael Gontijo Lopes · Alberto Marchisio · Erik Englesson · Wanqian Yang · Moritz Graule · Yi Sun · Daniel Kang · Mike Dusenberry · Min Du · Hartmut Maennel · Kunal Menda · Vineet Edupuganti · Luke Metz · David Stutz · Vignesh Srinivasan · Timo Sämann · Vineeth N Balasubramanian · Sina Mohseni · Rob Cornish · Judith Butepage · Zhangyang Wang · Bai Li · Bo Han · Honglin Li · Maksym Andriushchenko · Lukas Ruff · Meet P. Vadera · Yaniv Ovadia · Sunil Thulasidasan · Disi Ji · Gang Niu · Saeed Mahloujifar · Aviral Kumar · SANGHYUK CHUN · Dong Yin · Joyce Xu Xu · Hugo Gomes · Raanan Rohekar -
2019 Poster: Replica Conditional Sequential Monte Carlo »
Alex Shestopaloff · Arnaud Doucet -
2019 Poster: Scalable Metropolis-Hastings for Exact Bayesian Inference with Large Datasets »
Rob Cornish · Paul Vanetti · Alexandre Bouchard-Côté · George Deligiannidis · Arnaud Doucet -
2019 Oral: Replica Conditional Sequential Monte Carlo »
Alex Shestopaloff · Arnaud Doucet -
2019 Oral: Scalable Metropolis-Hastings for Exact Bayesian Inference with Large Datasets »
Rob Cornish · Paul Vanetti · Alexandre Bouchard-Côté · George Deligiannidis · Arnaud Doucet -
2019 Poster: On the Impact of the Activation function on Deep Neural Networks Training »
Soufiane Hayou · Arnaud Doucet · Judith Rousseau -
2019 Oral: On the Impact of the Activation function on Deep Neural Networks Training »
Soufiane Hayou · Arnaud Doucet · Judith Rousseau -
2018 Poster: On Nesting Monte Carlo Estimators »
Tom Rainforth · Rob Cornish · Hongseok Yang · andrew warrington · Frank Wood -
2018 Oral: On Nesting Monte Carlo Estimators »
Tom Rainforth · Rob Cornish · Hongseok Yang · andrew warrington · Frank Wood