Timezone: »

 
Poster
GFlowOut: Dropout with Generative Flow Networks
Dianbo Liu · Moksh Jain · Bonaventure F. P. Dossou · Qianli Shen · Salem Lahlou · Anirudh Goyal · Nikolay Malkin · Chris Emezue · Dinghuai Zhang · Nadhir Hassen · Xu Ji · Kenji Kawaguchi · Yoshua Bengio

Thu Jul 27 04:30 PM -- 06:00 PM (PDT) @ Exhibit Hall 1 #432
in Poster Session 6 »

Bayesian inference offers principled tools to tackle many critical problems with modern neural networks such as poor calibration and generalization, and data inefficiency. However, scaling Bayesian inference to large architectures is challenging and requires restrictive approximations. Monte Carlo Dropout has been widely used as a relatively cheap way to approximate inference and estimate uncertainty with deep neural networks. Traditionally, the dropout mask is sampled independently from a fixed distribution. Recent research shows that the dropout mask can be seen as a latent variable, which can be inferred with variational inference. These methods face two important challenges: (a) the posterior distribution over masks can be highly multi-modal which can be difficult to approximate with standard variational inference and (b) it is not trivial to fully utilize sample-dependent information and correlation among dropout masks to improve posterior estimation. In this work, we propose GFlowOut to address these issues. GFlowOut leverages the recently proposed probabilistic framework of Generative Flow Networks (GFlowNets) to learn the posterior distribution over dropout masks. We empirically demonstrate that GFlowOut results in predictive distributions that generalize better to out-of-distribution data and provide uncertainty estimates which lead to better performance in downstream tasks.

[ PDF]  [ Poster] 

Author Information

Dianbo Liu (Mila)
Moksh Jain (Mila / Université de Montréal)
Bonaventure F. P. Dossou (Mila, Google Research)
Bonaventure F. P. Dossou

I hold a Bachelor of Science in Mathematics and a Master of Science in Data Engineering. I am a drug discovery researcher at Mila Quebec AI Institute, working under the supervision of Professor Yoshua Bengio, and currently, I am also a Graduate Student at Google AI. My research areas include Machine & Deep learning (and its application in computer vision, natural language processing for Healthcare, and African Languages)

Qianli Shen (national university of singaore, National University of Singapore)
Salem Lahlou (Mila, Université de Montréal)
Anirudh Goyal (Université de Montréal)
Nikolay Malkin (Mila / Université de Montréal)
Chris Emezue (Technical University of Munich & Mila)
Dinghuai Zhang (Mila)
Nadhir Hassen (Université de Montréal)
Xu Ji (Montreal Institute for Learning Algorithms, University of Montreal, University of Montreal)
Kenji Kawaguchi (NUS)
Yoshua Bengio (Mila - Quebec AI Institute)

More from the Same Authors