Skip to yearly menu bar Skip to main content

Workshop: Workshop on Reinforcement Learning Theory

Sample Complexity of Offline Reinforcement Learning with Deep ReLU Networks

Tang Thanh Nguyen · Sunil Gupta · Hung Tran-The · Svetha Venkatesh


We study the statistical theory of offline reinforcement learning (RL) with deep ReLU network function approximation. We analyze a variant of fitted-Q iteration (FQI) algorithm under a new dynamic condition that we call Besov dynamic closure, which encompasses the conditions from prior analyses for deep neural network function approximation. Under Besov dynamic closure, we prove that the FQI-type algorithm enjoys an improved sample complexity than the prior results. Importantly, our sample complexity is obtained under the new general dynamic condition and a data-dependent structure where the latter is either ignored in prior algorithms or improperly handled by prior analyses. This is the first comprehensive analysis for offline RL with deep ReLU network function approximation under a general setting.

Chat is not available.