Timezone: »

 
Spotlight
Adapting the Linearised Laplace Model Evidence for Modern Deep Learning
Javier Antorán · David Janz · James Allingham · Erik Daxberger · Riccardo Barbano · Eric Nalisnick · Jose Miguel Hernandez-Lobato

Thu Jul 21 11:00 AM -- 11:05 AM (PDT) @ Room 310
in Probabilistic Methods/MISC »

The linearised Laplace method for estimating model uncertainty has received renewed attention in the Bayesian deep learning community. The method provides reliable error bars and admits a closed-form expression for the model evidence, allowing for scalable selection of model hyperparameters. In this work, we examine the assumptions behind this method, particularly in conjunction with model selection.We show that these interact poorly with some now-standard tools of deep learning--stochastic approximation methods and normalisation layers--and make recommendations for how to better adapt this classic method to the modern setting.We provide theoretical support for our recommendations and validate them empirically on MLPs, classic CNNs, residual networks with and without normalisation layers, generative autoencoders and transformers.

Keywords: Deep Learning · Probabilistic Methods · PM: Bayesian Models and Methods

Author Information

Javier Antorán (University of Cambridge)

I am a PhD student in Machine Learning at the University of Cambridge under the supervision of Dr. José Miguel Hernández-Lobato. I’m interested in Bayesian deep learning, representation learning, uncertainty in machine learning and information theory. I graduated from the University of Zaragoza in 2018 with an honorary distinction (“premio extraordinario”) in Telecommunications Engineering (EE/CS). I was awarded an MPhil in Machine Learning with distinction by the University of Cambridge in 2019. I also do freelance engineering consulting and am a co-founder of arisetech.es.

David Janz (University of Cambridge)
James Allingham (University of Cambridge)
Erik Daxberger (University of Cambridge & MPI for Intelligent Systems, Tübingen)
Riccardo Barbano (University College London)
Eric Nalisnick (University of Amsterdam)
Jose Miguel Hernandez-Lobato (University of Cambridge)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors