Evaluating LLMs When They Do Not Know the Answer: Statistical Evaluation of Mathematical Reasoning via Comparative Signals

Evaluating mathematical reasoning in LLMs is constrained by limited benchmark sizes and inherent model stochasticity, yielding high-variance accuracy estimates and unstable rankings across platforms. On difficult problems, an LLM may fail to produce a correct final answer, yet still provide reliable pairwise comparison signals indicating which of two candidate solutions is better. We leverage this observation to design a statistically efficient evaluation framework that combines standard labeled outcomes with pairwise comparison signals obtained by having models judge auxiliary reasoning chains. Treating these comparison signals as control variates, we develop a semiparametric estimator based on the efficient influence function (EIF) for the setting where auxiliary reasoning chains are observed. This yields a one-step estimator that achieves the semiparametric efficiency bound, guarantees strict variance reduction over naive sample averaging, and admits asymptotic normality for principled uncertainty quantification. Across simulations, our one-step estimator substantially improves ranking accuracy, with gains increasing as model output noise grows. Experiments on GPQA Diamond, AIME 2025, and GSM8K further demonstrate more precise performance estimation and more reliable model rankings, especially in small-sample regimes where conventional evaluation is pretty unstable.

Evaluating the mathematical reasoning ability of large language models is difficult because many benchmarks are small and model outputs can vary from run to run. As a result, a model’s reported accuracy, and even its ranking against other models, can change substantially depending on which questions are sampled or how the model responds. Our work starts from a simple observation: even when a model fails to produce the correct final answer, it may still be able to judge which of two candidate solutions is better. We use these comparison signals as extra information, together with standard correctness labels, to estimate model performance more reliably. Technically, we build a statistical estimator that combines final-answer accuracy with these auxiliary judgment signals to reduce uncertainty. This makes evaluations more sample-efficient: with the same number of benchmark questions, our method can produce more stable accuracy estimates and more reliable model rankings than simply averaging correct and incorrect answers. We demonstrate this benefit on mathematical reasoning benchmarks such as GPQA Diamond, AIME 2025, and GSM8K, where small sample sizes can make ordinary evaluation especially unstable.