THE CONFERENCE
FOR AUTHORS
FOR PARTICIPANTS

Timezone: »

 
Poster
SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient
Lam Nguyen · Jie Liu · Katya Scheinberg · Martin Takac

Tue Aug 08 01:30 AM -- 05:00 AM (PDT) @ Gallery #48
in Posters Tue »

In this paper, we propose a StochAstic Recursive grAdient algoritHm (SARAH), as well as its practical variant SARAH+, as a novel approach to the finite-sum minimization problems. Different from the vanilla SGD and other modern stochastic methods such as SVRG, S2GD, SAG and SAGA, SARAH admits a simple recursive framework for updating stochastic gradient estimates; when comparing to SAG/SAGA, SARAH does not require a storage of past gradients. The linear convergence rate of SARAH is proven under strong convexity assumption. We also prove a linear convergence rate (in the strongly convex case) for an inner loop of SARAH, the property that SVRG does not possess. Numerical experiments demonstrate the efficiency of our algorithm.

Author Information

Lam Nguyen (Lehigh University)
Jie Liu (Lehigh University)
Katya Scheinberg (Lehigh University)
Martin Takac (Lehigh University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors