Timezone: »

 
Spotlight
A Psychological Theory of Explainability
Scott Cheng-Hsin Yang · Nils Erik Tomas Folke · Patrick Shafto

Thu Jul 21 08:45 AM -- 08:50 AM (PDT) @ Room 307
in Theory/Social Aspects »

The goal of explainable Artificial Intelligence (XAI) is to generate human-interpretable explanations, but there are no computationally precise theories of how humans interpret AI generated explanations. The lack of theory means that validation of XAI must be done empirically, on a case-by-case basis, which prevents systematic theory-building in XAI. We propose a psychological theory of how humans draw conclusions from saliency maps, the most common form of XAI explanation, which for the first time allows for precise prediction of explainee inference conditioned on explanation. Our theory posits that absent explanation humans expect the AI to make similar decisions to themselves, and that they interpret an explanation by comparison to the explanations they themselves would give. Comparison is formalized via Shepard's universal law of generalization in a similarity space, a classic theory from cognitive science. A pre-registered user study on AI image classifications with saliency map explanations demonstrate that our theory quantitatively matches participants' predictions of the AI.

Keywords: T: Social Aspects · SA: Accountability, Transparency and Interpretability

Author Information

Scott Cheng-Hsin Yang (Rutgers University)
Nils Erik Tomas Folke (Rutgers)

I am a postdoctoral associate in the department of Mathematics and Computer Science at Rutgers University - Newark. I earned my Ph.D. from the University of Cambridge, studying the role of confidence in perceptual and value-based decisions. I then spent one year as a postdoc in Judge Business School at the University of Cambridge working on various projects related to how behavioral science can improve public policy. I extended this work at the Department of Policy and Management at Columbia University, where I investigated what factors predict preventive healthcare appointment attendance, before and during COVID-19, and how we can use machine learning to encourage people to attend. I am interested in how psychology and machine learning can be applied to improve real- world decision-making. I am particularly interested in how to build better decision-support systems by improving humans’ mental models of AI and improving AIs’ models of humans. The long-term aim of this work would be for a decision-support system to have a distinct model of every user, and for these models to update dynamically as the system learns more about each user.

Patrick Shafto (IAS / Rutgers University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors