Timezone: »

Risk-Sensitive Reinforcement Learning with Function Approximation: A Debiasing Approach
Yingjie Fei · Zhuoran Yang · Zhaoran Wang

Tue Jul 20 09:00 AM -- 11:00 AM (PDT) @

We study function approximation for episodic reinforcement learning with entropic risk measure. We first propose an algorithm with linear function approximation. Compared to existing algorithms, which suffer from improper regularization and regression biases, this algorithm features debiasing transformations in backward induction and regression procedures. We further propose an algorithm with general function approximation, which features implicit debiasing transformations. We prove that both algorithms achieve a sublinear regret and demonstrate a trade-off between generality and efficiency. Our analysis provides a unified framework for function approximation in risk-sensitive reinforcement learning, which leads to the first sublinear regret bounds in the setting.

Author Information

Yingjie Fei (Cornell University)
Zhuoran Yang (Princeton University)
Zhaoran Wang (Northwestern University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors