Skip to yearly menu bar Skip to main content


Good Initializations of Variational Bayes for Deep Models

Simone Rossi · Pietro Michiardi · Maurizio Filippone

Pacific Ballroom #83

Keywords: [ Bayesian Methods ] [ Bayesian Deep Learning ] [ Approximate Inference ]


Stochastic variational inference is an established way to carry out approximate Bayesian inference for deep models flexibly and at scale. While there have been effective proposals for good initializations for loss minimization in deep learning, far less attention has been devoted to the issue of initialization of stochastic variational inference. We address this by proposing a novel layer-wise initialization strategy based on Bayesian linear models. The proposed method is extensively validated on regression and classification tasks, including Bayesian Deep Nets and Conv Nets, showing faster and better convergence compared to alternatives inspired by the literature on initializations for loss minimization.

Live content is unavailable. Log in and register to view live content