Timezone: »

 
Discovering User Types: Characterization of User Traits by Task-Specific Behaviors in Reinforcement Learning
Lars L. Ankile · Brian Ham · Kevin Mao · Eura Shin · Siddharth Swaroop · Finale Doshi-Velez · Weiwei Pan

@
in Interactive Learning with Implicit Human Feedback »
Event URL: https://openreview.net/forum?id=c7mlZnxCKX »

We often want to infer user traits when personalizing interventions. Approaches like Inverse RL can learn traits formalized as parameters of a Markov Decision Process but are data intensive. Instead of inferring traits for individuals, we study the relationship between RL worlds and the set of user traits. We argue that understanding the breakdown of ``user types" within a world -- broad sets of traits that result in the same behavior -- helps rapidly personalize interventions. We show that seemingly different RL worlds admit the same set of user types and formalize this observation as an equivalence relation defined on worlds. We show that these equivalence classes capture many different worlds. We argue that the richness of these classes allows us to transfer insights on intervention design between toy and real worlds.

Author Information

Lars L. Ankile (Harvard University)
Brian Ham (Harvard University)
Kevin Mao
Eura Shin (Harvard University)
Siddharth Swaroop (Harvard University)
Finale Doshi-Velez (Harvard University)
Finale Doshi-Velez

Finale Doshi-Velez is a Gordon McKay Professor in Computer Science at the Harvard Paulson School of Engineering and Applied Sciences. She completed her MSc from the University of Cambridge as a Marshall Scholar, her PhD from MIT, and her postdoc at Harvard Medical School. Her interests lie at the intersection of machine learning, healthcare, and interpretability. Selected Additional Shinies: BECA recipient, AFOSR YIP and NSF CAREER recipient; Sloan Fellow; IEEE AI Top 10 to Watch

Weiwei Pan (Harvard University)

More from the Same Authors