Closed-loop Reasoning about Counterfactuals to Improve Policy Transparency

Explanations are a powerful way of increasing the transparency of complex policies. Such explanations must not only be informative regarding the policy in question, but must also be tailored to the human explainee. In particular, it is critical to consider the explainee's current beliefs and the counterfactuals (i.e. alternate outcomes) with which they will likely interpret any given explanation. E.g., the explainee will be inclined to wonder ``why did event P happen instead of counterfactual Q?'' In this vein, we first model human beliefs using a particle filter to consider the counterfactuals the human will likely use to interpret a potential explanation, which in turn helps select an explanation that is highly informative. Second, we design a closed-loop explanation framework, inspired by the education literature, that continuously updates the particle filter not only based on the explanations provided but also based on feedback from the human regarding their understanding. Finally, we present a user study design for testing the iterative modeling of a human's likely counterfactuals in conveying effective explanations.