Timezone: »

Stochastic modified equations and adaptive stochastic gradient algorithms
Qianxiao Li · Cheng Tai · Weinan E

Tue Aug 08 01:30 AM -- 05:00 AM (PDT) @ Gallery #71

We develop the method of stochastic modified equations (SME), in which stochastic gradient algorithms are approximated in the weak sense by continuous-time stochastic differential equations. We exploit the continuous formulation together with optimal control theory to derive novel adaptive hyper-parameter adjustment policies. Our algorithms have competitive performance with the added benefit of being robust to varying models and datasets. This provides a general methodology for the analysis and design of stochastic gradient algorithms.

Author Information

Qianxiao Li (Institute of High Performance Computing, A*STAR)
Cheng Tai (Peking University)
Weinan E (Princeton University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors