Timezone: »
Modern neural networks have proven to be powerful function approximators, providing state-of-the-art performance in a multitude of applications. They however fall short in their ability to quantify confidence in their predictions --- this is crucial in high-stakes applications that involve critical decision-making. Bayesian neural networks (BNNs) aim at solving this problem by placing a prior distribution over the network's parameters, thereby inducing a posterior distribution that encapsulates predictive uncertainty. While existing variants of BNNs based on Monte Carlo dropout produce reliable (albeit approximate) uncertainty estimates over in-distribution data, they tend to exhibit over-confidence in predictions made on target data whose feature distribution differs from the training data, i.e., the covariate shift setup. In this paper, we develop an approximate Bayesian inference scheme based on posterior regularisation, wherein unlabelled target data are used as ``pseudo-labels'' of model confidence that are used to regularise the model's loss on labelled source data. We show that this approach significantly improves the accuracy of uncertainty quantification on covariate-shifted data sets, with minimal modification to the underlying model architecture. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
Author Information
Alex Chan (University of Cambridge)
Ahmed Alaa (UCLA)
Zhaozhi Qian (University of Cambridge)
Mihaela van der Schaar (University of Cambridge and UCLA)
More from the Same Authors
-
2020 Poster: Discriminative Jackknife: Quantifying Uncertainty in Deep Learning via Higher-Order Influence Functions »
Ahmed Alaa · Mihaela van der Schaar -
2020 Poster: Time Series Deconfounder: Estimating Treatment Effects over Time in the Presence of Hidden Confounders »
Ioana Bica · Ahmed Alaa · Mihaela van der Schaar -
2020 Poster: Temporal Phenotyping using Deep Predictive Clustering of Disease Progression »
Changhee Lee · Mihaela van der Schaar -
2020 Poster: Learning for Dose Allocation in Adaptive Clinical Trials with Safety Constraints »
Cong Shen · Zhiyang Wang · Sofia Villar · Mihaela van der Schaar -
2020 Poster: Frequentist Uncertainty in Recurrent Neural Networks via Blockwise Influence Functions »
Ahmed Alaa · Mihaela van der Schaar -
2020 Poster: Inverse Active Sensing: Modeling and Understanding Timely Decision-Making »
Daniel Jarrett · Mihaela van der Schaar -
2020 Tutorial: Machine Learning for Healthcare: Challenges, Methods, Frontiers »
Mihaela van der Schaar -
2019 Poster: Validating Causal Inference Models via Influence Functions »
Ahmed Alaa · Mihaela van der Schaar -
2019 Oral: Validating Causal Inference Models via Influence Functions »
Ahmed Alaa · Mihaela van der Schaar -
2018 Poster: AutoPrognosis: Automated Clinical Prognostic Modeling via Bayesian Optimization with Structured Kernel Learning »
Ahmed M. Alaa Ibrahim · Mihaela van der Schaar -
2018 Oral: AutoPrognosis: Automated Clinical Prognostic Modeling via Bayesian Optimization with Structured Kernel Learning »
Ahmed M. Alaa Ibrahim · Mihaela van der Schaar -
2018 Poster: Limits of Estimating Heterogeneous Treatment Effects: Guidelines for Practical Algorithm Design »
Ahmed M. Alaa Ibrahim · Mihaela van der Schaar -
2018 Oral: Limits of Estimating Heterogeneous Treatment Effects: Guidelines for Practical Algorithm Design »
Ahmed M. Alaa Ibrahim · Mihaela van der Schaar -
2017 Poster: Learning from Clinical Judgments: Semi-Markov-Modulated Marked Hawkes Processes for Risk Prognosis »
Ahmed M. Alaa Ibrahim · Scott B Hu · Mihaela van der Schaar -
2017 Talk: Learning from Clinical Judgments: Semi-Markov-Modulated Marked Hawkes Processes for Risk Prognosis »
Ahmed M. Alaa Ibrahim · Scott B Hu · Mihaela van der Schaar