Conformal Reliability: A New Evaluation Metric for Conditional Generation

Conditional generative models have recently achieved remarkable success in various applications. However, a suitable metric for evaluating the reliability of these models, which takes into account their inherent uncertainty, is still lacking. Existing metrics, which typically assess a single output, may fail to capture the variability or potential risks in generation. In this paper, we propose a novel evaluation metric called reliability score based on conformal prediction, which measures the worst-case performance within the prediction set at a pre-specified confidence level. However, computing this score is challenging due to the high-dimensional nature of the output space and the nonconvexity of both the metric function and the prediction set. To efficiently compute this score, we introduce Conformal ReLiability (CReL), a framework that can (i) construct the prediction set with desired coverage; and (ii) accurately optimize the reliability score within the constructed prediction set. We provide theoretical results on coverage and demonstrate empirically that our method produces more informative prediction sets than existing approaches. Experiments on synthetic data and the image-to-text and text-to-image tasks further demonstrate the interpretability of our new metric, and the validity and effectiveness of our computational framework. Source code can be found at https://ggc29.github.io/CReL/.

Generative AI systems are usually evaluated by the quality of one output or by an average score, but the same input can lead to many plausible outputs, including bad or risky ones. This matters because in applications such as autonomous systems, scientific discovery, or safety-sensitive content generation, a model that looks good on average may still be unreliable if some likely outputs are misleading. We introduce Conformal ReLiability (CReL), a way to ask not only “how good is this model on average?” but also “how bad could its likely outputs be at a chosen confidence level?” CReL builds a statistically calibrated set of likely outputs and measures the lowest task-specific quality score within that set, while using a compact latent representation to make this computation practical for high-dimensional images and text. Experiments on synthetic data, image captioning, and text-to-image generation show that CReL can reveal reliability differences hidden by standard metrics and help identify models that maintain better worst-case performance.