Timezone: »

 
Poster
Hessian-Free High-Resolution Nesterov Acceleration For Sampling
Ruilin Li · Hongyuan Zha · Molei Tao

Tue Jul 19 03:30 PM -- 05:30 PM (PDT) @ Hall E #719
in Poster Session 1 »
Nesterov's Accelerated Gradient (NAG) for optimization has better performance than its continuous time limit (noiseless kinetic Langevin) when a finite step-size is employed (Shi et al., 2021). This work explores the sampling counterpart of this phenonemon and proposes a diffusion process, whose discretizations can yield accelerated gradient-based MCMC methods. More precisely, we reformulate the optimizer of NAG for strongly convex functions (NAG-SC) as a Hessian-Free High-Resolution ODE, change its high-resolution coefficient to a hyperparameter, inject appropriate noise, and discretize the resulting diffusion process. The acceleration effect of the new hyperparameter is quantified and it is not an artificial one created by time-rescaling. Instead, acceleration beyond underdamped Langevin in $W_2$ distance is quantitatively established for log-strongly-concave-and-smooth targets, at both the continuous dynamics level and the discrete algorithm level. Empirical experiments in both log-strongly-concave and multi-modal cases also numerically demonstrate this acceleration.
Keywords: PM: Monte Carlo and Sampling Methods

Author Information

Ruilin Li (Georgia Institute of Technology)
Hongyuan Zha (Shenzhen Institute of Artificial Intelligence and Robotics for Society; The Chinese University of Hong Kong, Shenzhen)
Molei Tao (Georgia Institute of Technology)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors