Optimal Decision-Making Based on Prediction Sets

Prediction sets can wrap around any ML model to cover unknown test outcomes with a guaranteed probability. Yet, it remains unclear how to use them optimally for downstream decision-making. Here, we propose a decision-theoretic framework that seeks to minimize the expected loss (risk) against a worst-case distribution consistent with the prediction set's coverage guarantee. We first characterize the minimax optimal policy for a fixed prediction set, showing that it balances the worst-case loss inside the set with a penalty for potential losses outside the set. Building on this, we derive the optimal prediction set construction that minimizes the resulting robust risk subject to a coverage constraint. Finally, we introduce Risk-Optimal Conformal Prediction (ROCP), a practical algorithm that targets these risk-minimizing sets while maintaining finite-sample distribution-free marginal coverage. Empirical evaluations on medical diagnosis and a toy static hazard-decision benchmark demonstrate that ROCP reduces critical mistakes compared to baselines, particularly when out-of-set errors are costly. The source code to reproduce our experiments is available at https://github.com/TaoWangPenn/Risk-Optimal-Conformal-Prediction.

Machine learning systems are often used to support decisions, such as choosing a medical treatment or reacting to a possible hazard. In these settings, the key question is not just whether a prediction is accurate, but whether the resulting action avoids costly mistakes. A common way to express uncertainty is to return a set of plausible outcomes instead of a single guess, but these sets do not by themselves tell us what action to take. We develop a framework for making decisions from such uncertainty sets in a way that guards against rare but high-cost errors. We also introduce a practical method, called Risk-Optimal Conformal Prediction (ROCP), that can be combined with many existing predictive models while still preserving a reliable coverage guarantee. In experiments on medical diagnosis and a toy hazard-decision task, our method reduces critical mistakes compared with standard alternatives, especially when rare errors are expensive.