Timezone: »

Revisiting the Effects of Stochasticity for Hamiltonian Samplers
Giulio Franzese · Dimitrios Milios · Maurizio Filippone · Pietro Michiardi

Thu Jul 21 03:00 PM -- 05:00 PM (PDT) @ #830
We revisit the theoretical properties of Hamiltonian stochastic differential equations (SDES) for Bayesian posterior sampling, and we study the two types of errors that arise from numerical SDE simulation: the discretization error and the error due to noisy gradient estimates in the context of data subsampling. Our main result is a novel analysis for the effect of mini-batches through the lens of differential operator splitting, revising previous literature results. The stochastic component of a Hamiltonian SDE is decoupled from the gradient noise, for which we make no normality assumptions.This leads to the identification of a convergence bottleneck: when considering mini-batches, the best achievable error rate is $\mathcal{O}(\eta^2)$, with $\eta$ being the integrator step size.Our theoretical results are supported by an empirical study on a variety of regression and classification tasks for Bayesian neural networks.

Author Information

Giulio Franzese (EURECOM)
Dimitrios Milios (EURECOM)
Maurizio Filippone (EURECOM)
Pietro Michiardi (EURECOM)

Pietro Michiardi received his M.S. in Computer Science from EURECOM and his M.S. in Electrical Engineering from Politecnico di Torino. Pietro received his Ph.D. in Computer Science from Telecom ParisTech (former ENST, Paris), and his HDR (Habilitation) from UNSA. Today, Pietro is a Professor of Computer Science at EURECOM, where he leads the Distributed System Group, which blends theory and system research focusing on large-scale distributed systems (including data processing and data storage), and scalable algorithm design to mine massive amounts of data. Additional research interests are on system, algorithmic, and performance evaluation aspects of distributed systems. Pietro has been appointed as Data Science department head in May 2016.

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors