Timezone: »
Bayesian neural networks (BNNs) promise improved generalization under covariate shift by providing principled probabilistic representations of epistemic uncertainty. However, weight-based BNNs often struggle with high computational complexity of large-scale architectures and datasets. Node-based BNNs have recently been introduced as scalable alternatives, which induce epistemic uncertainty by multiplying each hidden node with latent random variables, while learning a point-estimate of the weights. In this paper, we interpret these latent noise variables as implicit representations of simple and domain-agnostic data perturbations during training, producing BNNs that perform well under covariate shift due to input corruptions. We observe that the diversity of the implicit corruptions depends on the entropy of the latent variables, and propose a straightforward approach to increase the entropy of these variables during training. We evaluate the method on out-of-distribution image classification benchmarks, and show improved uncertainty estimation of node-based BNNs under covariate shift due to input perturbations. As a side effect, the method also provides robustness against noisy training labels.
Author Information
Trung Trinh (Aalto University)
Markus Heinonen (Aalto University)
Luigi Acerbi (University of Helsinki)

Assistant Professor at the Department of Computer Science, University of Helsinki. Member of the *Finnish Center for Artificial Intelligence* FCAI.
Samuel Kaski (Aalto University and University of Manchester)
Related Events (a corresponding poster, oral, or spotlight)
-
2022 Oral: Tackling covariate shift with node-based Bayesian neural networks »
Tue. Jul 19th 02:30 -- 02:50 PM Room Ballroom 3 & 4
More from the Same Authors
-
2023 : AbODE: Ab initio antibody design using conjoined ODEs »
Yogesh Verma · Markus Heinonen · Vikas K Garg -
2023 : AbODE: Ab initio antibody design using conjoined ODEs »
Yogesh Verma · Markus Heinonen · Vikas K Garg -
2023 : Augmenting Bayesian Optimization with Preference-based Expert Feedback »
Daolang Huang · Louis Filstroff · Petrus Mikkola · Runkai Zheng · Milica Todorovic · Samuel Kaski -
2023 : Bayesian Active Meta-Learning under Prior Misspecification »
Sabina Sloman · Ayush Bharti · Samuel Kaski -
2023 Poster: AbODE: Ab initio antibody design using conjoined ODEs »
Yogesh Verma · Markus Heinonen · Vikas K Garg -
2023 Poster: Optimally-weighted Estimators of the Maximum Mean Discrepancy for Likelihood-Free Inference »
Ayush Bharti · Masha Naslidnyk · Oscar Key · Samuel Kaski · Francois-Xavier Briol -
2022 Poster: Approximate Bayesian Computation with Domain Expert in the Loop »
Ayush Bharti · Louis Filstroff · Samuel Kaski -
2022 Spotlight: Approximate Bayesian Computation with Domain Expert in the Loop »
Ayush Bharti · Louis Filstroff · Samuel Kaski -
2021 Poster: Differentially Private Bayesian Inference for Generalized Linear Models »
Tejas Kulkarni · Joonas Jälkö · Antti Koskela · Samuel Kaski · Antti Honkela -
2021 Spotlight: Differentially Private Bayesian Inference for Generalized Linear Models »
Tejas Kulkarni · Joonas Jälkö · Antti Koskela · Samuel Kaski · Antti Honkela -
2021 Poster: Continuous-time Model-based Reinforcement Learning »
Cagatay Yildiz · Markus Heinonen · Harri Lähdesmäki -
2021 Spotlight: Continuous-time Model-based Reinforcement Learning »
Cagatay Yildiz · Markus Heinonen · Harri Lähdesmäki -
2020 Poster: Projective Preferential Bayesian Optimization »
Petrus Mikkola · Milica Todorović · Jari Järvi · Patrick Rinke · Samuel Kaski -
2019 : Networking Lunch (provided) + Poster Session »
Abraham Stanway · Alex Robson · Aneesh Rangnekar · Ashesh Chattopadhyay · Ashley Pilipiszyn · Benjamin LeRoy · Bolong Cheng · Ce Zhang · Chaopeng Shen · Christian Schroeder · Christian Clough · Clement DUHART · Clement Fung · Cozmin Ududec · Dali Wang · David Dao · di wu · Dimitrios Giannakis · Dino Sejdinovic · Doina Precup · Duncan Watson-Parris · Gege Wen · George Chen · Gopal Erinjippurath · Haifeng Li · Han Zou · Herke van Hoof · Hillary A Scannell · Hiroshi Mamitsuka · Hongbao Zhang · Jaegul Choo · James Wang · James Requeima · Jessica Hwang · Jinfan Xu · Johan Mathe · Jonathan Binas · Joonseok Lee · Kalai Ramea · Kate Duffy · Kevin McCloskey · Kris Sankaran · Lester Mackey · Letif Mones · Loubna Benabbou · Lynn Kaack · Matthew Hoffman · Mayur Mudigonda · Mehrdad Mahdavi · Michael McCourt · Mingchao Jiang · Mohammad Mahdi Kamani · Neel Guha · Niccolo Dalmasso · Nick Pawlowski · Nikola Milojevic-Dupont · Paulo Orenstein · Pedram Hassanzadeh · Pekka Marttinen · Ramesh Nair · Sadegh Farhang · Samuel Kaski · Sandeep Manjanna · Sasha Luccioni · Shuby Deshpande · Soo Kim · Soukayna Mouatadid · Sunghyun Park · Tao Lin · Telmo Felgueira · Thomas Hornigold · Tianle Yuan · Tom Beucler · Tracy Cui · Volodymyr Kuleshov · Wei Yu · yang song · Ydo Wexler · Yoshua Bengio · Zhecheng Wang · Zhuangfang Yi · Zouheir Malki -
2019 Poster: Active Learning for Decision-Making from Imbalanced Observational Data »
Iiris Sundin · Peter Schulam · Eero Siivola · Aki Vehtari · Suchi Saria · Samuel Kaski -
2019 Oral: Active Learning for Decision-Making from Imbalanced Observational Data »
Iiris Sundin · Peter Schulam · Eero Siivola · Aki Vehtari · Suchi Saria · Samuel Kaski -
2018 Poster: Learning unknown ODE models with Gaussian processes »
Markus Heinonen · Cagatay Yildiz · Henrik Mannerström · Jukka Intosalmi · Harri Lähdesmäki -
2018 Oral: Learning unknown ODE models with Gaussian processes »
Markus Heinonen · Cagatay Yildiz · Henrik Mannerström · Jukka Intosalmi · Harri Lähdesmäki -
2017 Workshop: Private and Secure Machine Learning »
Antti Honkela · Kana Shimizu · Samuel Kaski