Truthfulness Does Not Scale Like Reasoning: Why Polling Fails as a Proxy Verifier

Pass@$k$ and other methods of scaling inference compute can improve language model performance in domains with external verifiers, including mathematics and code, where incorrect candidates can be filtered reliably. This raises a natural question: can we similarly scale compute to elicit gains in truthfulness for domains without convenient verification? We show that across five benchmarks and models, surprisingly, it cannot. Even at $25\times$ the inference cost of naive sampling, polling-style aggregation yields no consistent accuracy gains over single-sample baselines and often amplifies correlated errors. We find that under uncertainty, models are better at predicting what other models will say within model ensembles than at identifying what is true, revealing a separation between social prediction and truth verification. Across models and benchmarks, aggregation fails to provide a robust truth signal because language model errors are strongly correlated. The source of correlation goes beyond any individual benchmark: we show that even when conditioned on out of distribution random strings and asked to produce pseudo-random outputs, different models produce correlated outputs. Confidence-based weighting provides no benefit because self-reported confidence fails to reliably distinguish correct from incorrect answers. These results delineate a boundary for inference-time scaling: in verified domains, additional samples provide more candidates for a verifier to filter; in unverified domains, additional samples merely reinforce correlated errors.

Many AI systems answer questions by generating several possible answers and then choosing the one that appears most often. This can work well for problems like math or programming, where answers can be checked. But many important questions do not have an easy way to verify the answer. We study whether asking many language models, or asking the same model many times, can make AI answers more truthful in these harder settings. Across several question-answering and forecasting tasks, we find that it usually does not. The models often agree with each other, but this agreement does not reliably mean they are correct. In many cases, the models make the same mistakes. Our results suggest that agreement among AI systems should not be treated as proof that an answer is true. It may still be useful as a warning sign that a question needs extra checking, such as looking up evidence, using tools, or asking a human. But for questions without a clear way to verify the answer, simply polling more AI systems is not enough to make them trustworthy.