Timezone: »

 
Poster
Does the Markov Decision Process Fit the Data: Testing for the Markov Property in Sequential Decision Making
Chengchun Shi · Runzhe Wan · Rui Song · Wenbin Lu · Ling Leng

Thu Jul 16 05:00 PM -- 05:45 PM & Fri Jul 17 04:00 AM -- 04:45 AM (PDT) @
in Poster Session 49 »

The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this paper, we propose a novel Forward-Backward Learning procedure to test MA in sequential decision making. The proposed test does not assume any parametric form on the joint distribution of the observed data and plays an important role for identifying the optimal policy in high-order Markov decision processes (MDPs) and partially observable MDPs. Theoretically, we establish the validity of our test. Empirically, we apply our test to both synthetic datasets and a real data example from mobile health studies to illustrate its usefulness.

Keywords: Healthcare · Reinforcement Learning · Reinforcement Learning Theory · Reinforcement Learning - General

Author Information

Chengchun Shi (London School of Economics and Political Science)
Runzhe Wan (North Carolina State University)
Rui Song (North Carolina State University)
Wenbin Lu (North Carolina State University)
Ling Leng (Amazon)

More from the Same Authors