Timezone: »
Physics-informed neural networks (PINNs) are computationally efficient alternatives to traditional partial differential equation (PDE) solvers.However, their reliability is dependent on the accuracy of the trained neural network. We introduce a mechanism for leveraging the symmetries of a given PDE to improve the neural solver. In particular, we propose a loss function that informs the network about Lie point symmetries in the same way that PINN models try to enforce the underlying PDE. Intuitively, our symmetry loss tries to ensure that infinitesimal generators of the Lie group preserve solutions of the PDE. This means that once the network learns a solution, it also learns the neighbouring solutions generated by Lie point symmetries.Our results show that symmetry is an effective inductive bias for PINNs and lead to a significant increase in sample efficiency.
Author Information
Tara Akhound-Sadegh (McGill-Mila)
Laurence Perreault-Levasseur (Université de Montréal and Mila)
Laurence Perreault Levasseur is an assistant professor at university of Montreal and Mila. She specializes in the development of machine learning methods for the analysis of cosmological data, with an emphasis on strong lensing data analysis. She is particularly interested in difficult inference problems, in particular in high dimension. She is the Canada research chair in Computational Cosmology and and Artificial Intelligence
Johannes Brandstetter (Microsoft)
Max Welling (University of Amsterdam & Qualcomm)
Siamak Ravanbakhsh (McGill - Mila)
More from the Same Authors
-
2022 : Galaxies on graph neural networks: towards robust synthetic galaxy catalogs with deep generative models »
Yesukhei Jagvaral · Rachel Mandelbaum · Francois Lanusse · Siamak Ravanbakhsh · Sukhdeep Singh · Duncan Campbell -
2022 : Path Integral Stochastic Optimal Control for Sampling Transition Paths »
Lars Holdijk · Yuanqi Du · Priyank Jaini · Ferry Hooft · Bernd Ensing · Max Welling -
2023 : Towards Unbiased Gravitational-Wave Parameter Estimation using Score-Based Likelihood Characterization »
Ronan Legin · Kaze Wong · Maximiliano Isi · Alexandre Adam · Laurence Perreault-Levasseur · Yashar Hezaveh -
2023 : ClimaX: A Foundation Model for Weather and Climate »
Tung Nguyen · Johannes Brandstetter · Ashish Kapoor · Jayesh K. Gupta · Aditya Grover -
2023 : Time Delay Cosmography with a Neural Ratio Estimator »
Ève Campeau-Poirier · Laurence Perreault-Levasseur · Adam Coogan · Yashar Hezaveh -
2023 Workshop: 2nd ICML Workshop on Machine Learning for Astrophysics »
Francois Lanusse · Marc Huertas-Company · Brice Menard · Laurence Perreault-Levasseur · J. Xavier Prochaska · Uros Seljak · Francisco Villaescusa-Navarro · Ashley Villar -
2023 : Modeling Accurate Long Rollouts with Temporal Neural PDE Solvers »
Phillip Lippe · Bastiaan Veeling · Paris Perdikaris · Richard E Turner · Johannes Brandstetter -
2023 Workshop: Structured Probabilistic Inference and Generative Modeling »
Dinghuai Zhang · Yuanqi Du · Chenlin Meng · Shawn Tan · Yingzhen Li · Max Welling · Yoshua Bengio -
2023 : Opening Remark »
Dinghuai Zhang · Yuanqi Du · Chenlin Meng · Shawn Tan · Yingzhen Li · Max Welling · Yoshua Bengio -
2023 Poster: Equivariance with Learned Canonicalization Functions »
Sékou-Oumar Kaba · Arnab Kumar Mondal · Yan Zhang · Yoshua Bengio · Siamak Ravanbakhsh -
2023 Poster: Sampling-Based Accuracy Testing of Posterior Estimators for General Inference »
Pablo Lemos · Adam Coogan · Yashar Hezaveh · Laurence Perreault-Levasseur -
2023 Poster: Neural Wave Machines: Learning Spatiotemporally Structured Representations with Locally Coupled Oscillatory Recurrent Neural Networks »
T. Anderson Keller · Max Welling -
2023 Poster: ClimaX: A foundation model for weather and climate »
Tung Nguyen · Johannes Brandstetter · Ashish Kapoor · Jayesh K. Gupta · Aditya Grover -
2023 Poster: Latent Traversals in Generative Models as Potential Flows »
Yue Song · T. Anderson Keller · Nicu Sebe · Max Welling -
2023 Poster: Geometric Clifford Algebra Networks »
David Ruhe · Jayesh K. Gupta · Steven De Keninck · Max Welling · Johannes Brandstetter -
2022 : Machine Learning for Scientific Discovery »
Josh Bloom · Daniela Huppenkothen · Laurence Perreault-Levasseur · George Stein · Francisco Villaescusa-Navarro -
2022 : GaMPEN: An ML Framework for Estimating Galaxy Morphological Parameters and Quantifying Uncertainty »
Aritra Ghosh · C. Megan Urry · Amrit Rau · Laurence Perreault-Levasseur -
2022 Poster: EqR: Equivariant Representations for Data-Efficient Reinforcement Learning »
Arnab Kumar Mondal · Vineet Jain · Kaleem Siddiqi · Siamak Ravanbakhsh -
2022 Poster: Utility Theory for Sequential Decision Making »
Mehran Shakerinava · Siamak Ravanbakhsh -
2022 Poster: Lie Point Symmetry Data Augmentation for Neural PDE Solvers »
Johannes Brandstetter · Max Welling · Daniel Worrall -
2022 Spotlight: Utility Theory for Sequential Decision Making »
Mehran Shakerinava · Siamak Ravanbakhsh -
2022 Spotlight: EqR: Equivariant Representations for Data-Efficient Reinforcement Learning »
Arnab Kumar Mondal · Vineet Jain · Kaleem Siddiqi · Siamak Ravanbakhsh -
2022 Spotlight: Lie Point Symmetry Data Augmentation for Neural PDE Solvers »
Johannes Brandstetter · Max Welling · Daniel Worrall -
2022 Poster: Equivariant Diffusion for Molecule Generation in 3D »
Emiel Hoogeboom · Victor Garcia Satorras · Clément Vignac · Max Welling -
2022 Oral: Equivariant Diffusion for Molecule Generation in 3D »
Emiel Hoogeboom · Victor Garcia Satorras · Clément Vignac · Max Welling -
2021 Poster: Equivariant Networks for Pixelized Spheres »
Mehran Shakerinava · Siamak Ravanbakhsh -
2021 Test Of Time: Bayesian Learning via Stochastic Gradient Langevin Dynamics »
Yee Teh · Max Welling -
2021 Test Of Time: Test of Time Award »
Max Welling · Max Welling -
2021 Spotlight: Equivariant Networks for Pixelized Spheres »
Mehran Shakerinava · Siamak Ravanbakhsh -
2021 Poster: The Hintons in your Neural Network: a Quantum Field Theory View of Deep Learning »
Roberto Bondesan · Max Welling -
2021 Spotlight: The Hintons in your Neural Network: a Quantum Field Theory View of Deep Learning »
Roberto Bondesan · Max Welling -
2021 Poster: A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups »
Marc Finzi · Max Welling · Andrew Wilson -
2021 Oral: A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups »
Marc Finzi · Max Welling · Andrew Wilson -
2021 Poster: Federated Learning of User Verification Models Without Sharing Embeddings »
Hossein Hosseini · Hyunsin Park · Sungrack Yun · Christos Louizos · Joseph B Soriaga · Max Welling -
2021 Poster: E(n) Equivariant Graph Neural Networks »
Victor Garcia Satorras · Emiel Hoogeboom · Max Welling -
2021 Poster: Self Normalizing Flows »
T. Anderson Keller · Jorn Peters · Priyank Jaini · Emiel Hoogeboom · Patrick Forré · Max Welling -
2021 Spotlight: E(n) Equivariant Graph Neural Networks »
Victor Garcia Satorras · Emiel Hoogeboom · Max Welling -
2021 Spotlight: Federated Learning of User Verification Models Without Sharing Embeddings »
Hossein Hosseini · Hyunsin Park · Sungrack Yun · Christos Louizos · Joseph B Soriaga · Max Welling -
2021 Spotlight: Self Normalizing Flows »
T. Anderson Keller · Jorn Peters · Priyank Jaini · Emiel Hoogeboom · Patrick Forré · Max Welling -
2020 : Invited talk 1: Unifying VAEs and Flows »
Max Welling -
2020 Poster: Universal Equivariant Multilayer Perceptrons »
Siamak Ravanbakhsh -
2020 Poster: Involutive MCMC: a Unifying Framework »
Kirill Neklyudov · Max Welling · Evgenii Egorov · Dmitry Vetrov -
2019 Workshop: Learning and Reasoning with Graph-Structured Representations »
Ethan Fetaya · Zhiting Hu · Thomas Kipf · Yujia Li · Xiaodan Liang · Renjie Liao · Raquel Urtasun · Hao Wang · Max Welling · Eric Xing · Richard Zemel -
2019 : Poster discussion »
Roman Novak · Maxime Gabella · Frederic Dreyer · Siavash Golkar · Anh Tong · Irina Higgins · Mirco Milletari · Joe Antognini · Sebastian Goldt · Adín Ramírez Rivera · Roberto Bondesan · Ryo Karakida · Remi Tachet des Combes · Michael Mahoney · Nicholas Walker · Stanislav Fort · Samuel Smith · Rohan Ghosh · Aristide Baratin · Diego Granziol · Stephen Roberts · Dmitry Vetrov · Andrew Wilson · César Laurent · Valentin Thomas · Simon Lacoste-Julien · Dar Gilboa · Daniel Soudry · Anupam Gupta · Anirudh Goyal · Yoshua Bengio · Erich Elsen · Soham De · Stanislaw Jastrzebski · Charles H Martin · Samira Shabanian · Aaron Courville · Shorato Akaho · Lenka Zdeborova · Ethan Dyer · Maurice Weiler · Pim de Haan · Taco Cohen · Max Welling · Ping Luo · zhanglin peng · Nasim Rahaman · Loic Matthey · Danilo J. Rezende · Jaesik Choi · Kyle Cranmer · Lechao Xiao · Jaehoon Lee · Yasaman Bahri · Jeffrey Pennington · Greg Yang · Jiri Hron · Jascha Sohl-Dickstein · Guy Gur-Ari -
2019 : Panel Discussion (moderator: Tom Dietterich) »
Max Welling · Kilian Weinberger · Terrance Boult · Dawn Song · Thomas Dietterich -
2019 : Keynote by Max Welling: A Nonparametric Bayesian Approach to Deep Learning (without GPs) »
Max Welling -
2019 Workshop: Joint Workshop on On-Device Machine Learning & Compact Deep Neural Network Representations (ODML-CDNNR) »
Sujith Ravi · Zornitsa Kozareva · Lixin Fan · Max Welling · Yurong Chen · Werner Bailer · Brian Kulis · Haoji Hu · Jonathan Dekhtiar · Yingyan Lin · Diana Marculescu -
2019 Workshop: Theoretical Physics for Deep Learning »
Jaehoon Lee · Jeffrey Pennington · Yasaman Bahri · Max Welling · Surya Ganguli · Joan Bruna -
2019 : Opening Remarks »
Jaehoon Lee · Jeffrey Pennington · Yasaman Bahri · Max Welling · Surya Ganguli · Joan Bruna -
2019 Poster: Stochastic Beams and Where To Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement »
Wouter Kool · Herke van Hoof · Max Welling -
2019 Oral: Stochastic Beams and Where To Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement »
Wouter Kool · Herke van Hoof · Max Welling -
2019 Poster: Emerging Convolutions for Generative Normalizing Flows »
Emiel Hoogeboom · Rianne Van den Berg · Max Welling -
2019 Oral: Emerging Convolutions for Generative Normalizing Flows »
Emiel Hoogeboom · Rianne Van den Berg · Max Welling -
2019 Poster: Gauge Equivariant Convolutional Networks and the Icosahedral CNN »
Taco Cohen · Maurice Weiler · Berkay Kicanaoglu · Max Welling -
2019 Oral: Gauge Equivariant Convolutional Networks and the Icosahedral CNN »
Taco Cohen · Maurice Weiler · Berkay Kicanaoglu · Max Welling -
2018 Poster: Attention-based Deep Multiple Instance Learning »
Maximilian Ilse · Jakub Tomczak · Max Welling -
2018 Oral: Attention-based Deep Multiple Instance Learning »
Maximilian Ilse · Jakub Tomczak · Max Welling -
2018 Invited Talk: Intelligence per Kilowatthour »
Max Welling -
2018 Poster: Neural Relational Inference for Interacting Systems »
Thomas Kipf · Ethan Fetaya · Kuan-Chieh Wang · Max Welling · Richard Zemel -
2018 Poster: BOCK : Bayesian Optimization with Cylindrical Kernels »
ChangYong Oh · Efstratios Gavves · Max Welling -
2018 Oral: Neural Relational Inference for Interacting Systems »
Thomas Kipf · Ethan Fetaya · Kuan-Chieh Wang · Max Welling · Richard Zemel -
2018 Oral: BOCK : Bayesian Optimization with Cylindrical Kernels »
ChangYong Oh · Efstratios Gavves · Max Welling -
2017 Poster: Multiplicative Normalizing Flows for Variational Bayesian Neural Networks »
Christos Louizos · Max Welling -
2017 Talk: Multiplicative Normalizing Flows for Variational Bayesian Neural Networks »
Christos Louizos · Max Welling -
2017 Poster: Equivariance Through Parameter-Sharing »
Siamak Ravanbakhsh · Jeff Schneider · Barnabás Póczos -
2017 Talk: Equivariance Through Parameter-Sharing »
Siamak Ravanbakhsh · Jeff Schneider · Barnabás Póczos