Statistical Testing of Large Language Model Reasoning

Improvements in the linguistic abilities of Large Language Models have made it increasingly difficult to distinguish between correct or incorrect answers they might provide in response to a query. In response, many so-called confidence measures have been developed that quantify the degree of confidence an LLM has in an answer based on a given reasoning, at little to no additional cost beyond what was needed to generate the answer. Given such a measure, the answer is then presented to the user only if the confidence is above a certain pre-defined threshold. In this work, we argue that such a procedure can be reinterpreted as a statistical test based on a single reasoning sample, and to improve its power, we propose instead to generalize the test to a mean of multiple samples. To keep the additional computation reasonable, we propose a novel procedure called StepBootstrap which generates alternative reasoning paths at low cost by resampling steps. We show that its confidence measures capture well the ideal distribution, from which we derive a more powerful test while keeping the computational overhead low.