Timezone: »

 
Poster
Clinician-in-the-Loop Decision Making: Reinforcement Learning with Near-Optimal Set-Valued Policies
Shengpu Tang · Aditya Modi · Michael Sjoding · Jenna Wiens

Thu Jul 16 07:00 AM -- 07:45 AM & Thu Jul 16 06:00 PM -- 06:45 PM (PDT) @ Virtual
in Poster Session 42 »

Standard reinforcement learning (RL) aims to find an optimal policy that identifies the best action for each state. However, in healthcare settings, many actions may be near-equivalent with respect to the reward (e.g., survival). We consider an alternative objective -- learning set-valued policies to capture near-equivalent actions that lead to similar cumulative rewards. We propose a model-free algorithm based on temporal difference learning and a near-greedy heuristic for action selection. We analyze the theoretical properties of the proposed algorithm, providing optimality guarantees and demonstrate our approach on simulated environments and a real clinical task. Empirically, the proposed algorithm exhibits good convergence properties and discovers meaningful near-equivalent actions. Our work provides theoretical, as well as practical, foundations for clinician/human-in-the-loop decision making, in which humans (e.g., clinicians, patients) can incorporate additional knowledge (e.g., side effects, patient preference) when selecting among near-equivalent actions.

Keywords: Healthcare · Reinforcement Learning · Applications - Neuroscience, Cognitive Science, Biology and Health
[ Slides] 

Author Information

Shengpu Tang (University of Michigan)
Shengpu Tang

Shengpu Tang is a PhD candidate in the computer science department at the University of Michigan. He is a member of the Machine Learning for Data-Driven Decisions (MLD3) research group led by Jenna Wiens. His current research focuses on developing computational methods that help solve important problems in healthcare, such as risk stratification and dynamic treatment recommendations. More generally, he is interested in broader applications of AI/ML, reinforcement learning and graph mining, computer game design, self-driving cars, security and hacking, as well as teaching. For more details, see his website at https://shengpu-tang.me/

Aditya Modi (University of Michigan)
Michael Sjoding (University of Michigan)
Jenna Wiens (University of Michigan)

More from the Same Authors