Can LLMs deliberate? Benchmarking Collective Reasoning for Democratic AI Applications

Multi-agent LLM systems are increasingly proposed for democratic applications, including consensus-finding, deliberation moderation, and the representation of stakeholder perspectives. While existing benchmarks for collective reasoning in LLM systems show strong performance on verifiable tasks such as Maths problems, we argue that these benchmarks cannot assess deliberative reasoning on contested normative questions, which, however, is exactly what democratic applications demand. We introduce DelibSim, a configurable simulation environment that benchmarks multi-agent deliberation along two theoretically grounded dimensions: procedural discourse quality (AQuA) and deliberative reasoning quality (DRI). Across 1,980 five-agent deliberations spanning 11 model configurations and 12 citizen-assembly topics, LLMs qchieve discourse quality statistically indistinguishable from human deliberation (AQuA 2.94 vs. 2.98). Normative prompting yields a small but reliable improvement in shared understanding (∆DRI = 0.029, p = 0.005), but effects do not survive topic-level correction and turn negative on ethically complex topics. Most strikingly, LLM groups exhibit far lower perspective diversity than human groups (6.5 vs. 18.8) and reversed convergence dynamics: human deliberation decreases dispersion as diverse views synthesize, whereas LLM deliberation increases it. These findings expose an important failure mode for AI deployed in deliberative settings: high-quality deliberative discourse can mask fundamentally different reasoning dynamics. We release DelibSim as an open benchmark to support the responsible deployment of multiagent AI in democratic applications.