Timezone: »

 
Bayesian Neural Networks with Domain Knowledge
Dylan Sam · Rattana Pukdee · Daniel Jeong · Yewon Byun · Zico Kolter

@
in Knowledge and Logical Reasoning in the Era of Data-driven Learning »

Prior knowledge about particular domains can help inform deep learning models to perform better and exhibit desirable behavior, combatting some of the issues with unfair or biased datasets. In this paper, we propose a general framework via variational inference to incorporate such prior information into Bayesian neural networks (BNNs). We learn an informative prior over neural network weights that assigns high probability mass to neural network weights that capture our domain knowledge, leading to a predictor (through posterior averaging) that also exhibits this behavior. We demonstrate that this approach improves upon standard BNNs and is comparable to frequentist approaches across many datasets with different types of prior information, including fairness, physics rules, and healthcare knowledge.

Author Information

Dylan Sam (Carnegie Mellon University)
Rattana Pukdee (Carnegie Mellon University)
Daniel Jeong (Carnegie Mellon University)
Yewon Byun (Carnegie Mellon University)
Zico Kolter (Carnegie Mellon University / Bosch Center for AI)

More from the Same Authors