WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingales

Responsibly deploying artificial intelligence (AI) / machine learning (ML) systems in high-stakes settings arguably requires not only proof of system reliability, but also continual, post-deployment monitoring to quickly detect and address any unsafe behavior. Methods for nonparametric sequential testing---especially conformal test martingales (CTMs) and anytime-valid inference---offer promising tools for this monitoring task. However, existing approaches are restricted to monitoring limited hypothesis classes or ``alarm criteria'' (e.g., detecting data shifts that violate certain exchangeability or IID assumptions), do not allow for online adaptation in response to shifts, and/or cannot diagnose the cause of degradation or alarm. In this paper, we address these limitations by proposing a weighted generalization of conformal test martingales (WCTMs), which lay a theoretical foundation for online monitoring for any unexpected changepoints in the data distribution while controlling false-alarms. For practical applications, we propose specific WCTM algorithms that adapt online to mild covariate shifts (in the marginal input distribution), quickly detect harmful shifts, and diagnose those harmful shifts as concept shifts (in the conditional label distribution) or extreme (out-of-support) covariate shifts that cannot be easily adapted to. On real-world datasets, we demonstrate improved performance relative to state-of-the-art baselines.

Responsibly deploying artificial intelligence (AI) systems in high-stakes settings—such as to assist doctors in medical diagnosis or AI chatbot agents used at internet-wide scale—requires continual monitoring of the AI’s performance, to quickly detect and address any unsafe behavior. It turns out, however, that AI monitoring is very difficult: many current approaches are too slow to detect truly harmful changes, or they may raise many false alarms and so risk creating a “boy who cried wolf” effect.To address these challenges, we introduce a novel framework called “WATCH” (Weighted Adaptive Testing for Changepoint Hypotheses), which monitors AI deployments with three main goals:    (1) Detection: First, WATCH quickly detects harmful shifts, and it catches them faster than directly tracking standard risk metrics.    (2) Adaptation: Second, WATCH adapts in real-time to mild or benign changes (which are commonplace but not harmful) to reduce unnecessary alarms and improve the utility of the AI’s outputs for end-users.    (3) Root-Cause Analysis: Lastly, WATCH diagnoses the cause of degradation (ie, as a severe change in the data inputs or a fundamental “concept shift” in input-output relationships) to inform appropriate recovery (eg, retraining) of the AI.WATCH charts a path toward safer and more responsible deployment of AI systems via adaptive monitoring.