Off-Policy Evaluation with Strategic Agents via Local Disclosure

We study off-policy evaluation (OPE) under strategic behavior where decision subjects (or agents) respond to a decision maker's policy by strategically modifying their covariates. Such behavior induces a policy-dependent covariate shift, breaking the standard assumption in existing methods that covariates are exogenous to the policy. Related work addresses this challenge by imposing strong assumptions such as repeated interactions or full knowledge of agents’ response behavior, substantially limiting its applicability to OPE. In contrast, we consider a one-shot OPE setting where the decision maker has only partial knowledge of the agents' response behavior. Our key insight is that disclosing local information through post-hoc explanations reveals agents’ pre-strategic covariates prior to adaptation, mitigating the information loss induced by strategic behavior. Leveraging this structure, we estimate a statistical model for the agents’ responses and construct a doubly robust estimator for policy value. By assuming that the agents' cost sensitivity follows a conditional log-normal distribution, we establish consistency of the proposed estimator and validate our approach empirically. More broadly, our results highlight how interaction design can mitigate information asymmetry by revealing otherwise hidden structure in agents' strategic responses.

Real-world decision making systems, e.g., lending or admissions policies, are evaluated using historical data before they are deployed. However, existing evaluation methods typically assume that people do not adapt their profiles in response to the policy itself. In reality, people may strategically change their observable characteristics to receive more favorable decisions, causing the population affected by the policy to change. This makes it difficult to reliably estimate how a new policy will perform before deployment. We study how to evaluate decision policies when people behave strategically and when their exact behavioral preferences are unknown. Our approach uses personalized feedback, such as post-hoc explanations, to reveal information about individuals prior to their strategic adaptation. Using this additional information, we estimate how people respond to policy changes and construct a robust method for evaluating new policies from historical data. Our results show that the design of interactions between decision makers and individuals can reduce information asymmetry and improve the reliability of policy evaluation in strategic settings.