Efficient Bayesian Inference from Noisy Pairwise Comparisons

Evaluating generative models is challenging because standard metrics often fail to reflect human preferences. Human evaluations are more reliable but costly and noisy, as participants vary in expertise, attention, and diligence. Pairwise comparisons improve consistency, yet aggregating them into overall quality scores requires careful modeling. Bradley-Terry-based methods update item scores from comparisons, but existing approaches either ignore rater variability or lack convergence guarantees, limiting robustness and interpretability. We introduce BBQ, a Bayesian Bradley-Terry variant that explicitly models rater quality, downweighting or removing unreliable participants, and provides guaranteed monotonic likelihood convergence through an Expectation-Maximization algorithm. Empirical results show that BBQ provides efficient inference, well-calibrated uncertainty estimates, and more robust, interpretable rankings compared to baseline Bradley-Terry models, even with noisy or crowdsourced raters. This framework enables more reliable and cost-effective human evaluation of generative models.

It is often hard to tell which AI system is better, especially for tasks like writing text or generating images, where automatic scores do not always match what people actually prefer. A common solution is to ask people to compare two outputs and choose the better one. However, these judgments can be noisy because some people are more careful or consistent than others. In this paper, we introduce a method that combines many of these pair-by-pair judgments while taking into account that some responses may be less reliable. This makes the final rankings more stable, more interpretable, and less sensitive to noisy evaluations. Our results show that the method works well across several datasets and can produce useful uncertainty estimates, showing when differences between systems are meaningful and when they are not. Overall, this work can help make human evaluation of AI systems more reliable, efficient, and trustworthy.