Timezone: »

 
Oral
Good Initializations of Variational Bayes for Deep Models
Simone Rossi · Pietro Michiardi · Maurizio Filippone

Thu Jun 13 04:25 PM -- 04:30 PM (PDT) @ Grand Ballroom
in Bayesian Deep Learning »

Stochastic variational inference is an established way to carry out approximate Bayesian inference for deep models. While there have been effective proposals for good initializations for loss minimization in deep learning, far less attention has been devoted to the issue of initialization of stochastic variational inference. We address this by proposing a novel layer-wise initialization strategy based on Bayesian linear models. The proposed method is extensively validated on regression and classification tasks, including Bayesian DeepNets and ConvNets, showing faster and better convergence compared to alternatives inspired by the literature on initializations for loss minimization.

[ Slides] 

Author Information

Simone Rossi (EURECOM)

PhD Student in Bayesian Deep Learning at Sorbonne Universite/EURECOM (previously MSc in Electronic Engineering and MSc in Computer Engineering). Under the supervision of Prof. Maurizio Filippone, I'm investigating new and exciting problems in Deep Probabilistic Modelling, Approximate Inference, Bayesian Deep Learning and Variational Inference.

Pietro Michiardi (EURECOM)

Pietro Michiardi received his M.S. in Computer Science from EURECOM and his M.S. in Electrical Engineering from Politecnico di Torino. Pietro received his Ph.D. in Computer Science from Telecom ParisTech (former ENST, Paris), and his HDR (Habilitation) from UNSA. Today, Pietro is a Professor of Computer Science at EURECOM, where he leads the Distributed System Group, which blends theory and system research focusing on large-scale distributed systems (including data processing and data storage), and scalable algorithm design to mine massive amounts of data. Additional research interests are on system, algorithmic, and performance evaluation aspects of distributed systems. Pietro has been appointed as Data Science department head in May 2016.

Maurizio Filippone (Eurecom)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors