Skip to yearly menu bar Skip to main content


Online Variance Reduction with Mixtures

Zal├ín Borsos · Sebastian Curi · Yehuda Levy · Andreas Krause

Pacific Ballroom #157

Keywords: [ Optimization - Others ] [ Online Learning ]


Adaptive importance sampling for stochastic optimization is a promising approach that offers improved convergence through variance reduction. In this work, we propose a new framework for variance reduction that enables the use of mixtures over predefined sampling distributions, which can naturally encode prior knowledge about the data. While these sampling distributions are fixed, the mixture weights are adapted during the optimization process. We propose VRM, a novel and efficient adaptive scheme that asymptotically recovers the best mixture weights in hindsight and can also accommodate sampling distributions over sets of points. We empirically demonstrate the versatility of VRM in a range of applications.

Live content is unavailable. Log in and register to view live content