Oral
Hessian Aided Policy Gradient
Zebang Shen · Alejandro Ribeiro · Hamed Hassani · Hui Qian · Chao Mi

Wed Jun 12th 04:30 -- 04:35 PM @ Room 104
in Reinforcement Learning Theory » 
Reducing the variance of estimators for policy gradient has long been the focus of reinforcement learning research.
While classic algorithms like REINFORCE find an $\epsilon$-approximate first-order stationary point in $\OM({1}/{\epsilon^4})$ random trajectory simulations, no provable  improvement on the complexity has been made so far.
This paper presents a Hessian aided policy gradient method with the first improved sample complexity of $\OM({1}/{\epsilon^3})$.
While our method exploits information from the policy Hessian, it can be implemented in linear time with respect to the parameter dimension and is hence applicable to sophisticated DNN parameterization.
Simulations on standard tasks validate the efficiency of our method.

Author Information

Zebang Shen (Zhejiang University)
Alejandro Ribeiro (University of Pennsylvania)
Hamed Hassani (University of Pennsylvania)
Hui Qian (Zhejiang University)
Chao Mi (Zhejiang University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors