Do LLMs Acknowledge Disputed Facts? A Benchmark for Factual Pluralism in LLMs
Enfa Fane ⋅ Mihai Surdeanu
Abstract
Pluralistic alignment research has focused predominantly on subjective and preference-based tasks, yet factual questions too can have multiple legitimate answers depending on whose sources and authorities one recognises. Models that assert a single value with confidence, rather than acknowledging such plurality, risk the same failures of representation that motivate pluralistic alignment in subjective settings. To address this gap, we introduce a dataset of 1,139 disputed factual claims from Wikidata and evaluate four LLMs on whether they acknowledge dispute. No model does so reliably, and reasoning mode hurts more than it helps. We release our dataset and code to support future work on factual pluralism in LLM evaluation.
Successful Page Load