The Persona Fidelity Gap: Behaviorally Grounded LLM Personas Still Compress Real-User Preference Diversity

Large language models (LLMs) increasingly stand in for diverse human users in preference data collection, recommender evaluation, alignment research, and social-science simulation. Whether LLM personas reproduce real preference distributions has not been measured at scale, leaving downstream training, evaluation, and policy decisions exposed to silent bias. Prior persona work either skips behavioral validation or treats persona variability as a tunable hyperparameter without calibrating to a real distribution. We learn a preference embedding for roughly 268K Amazon-Reviews users via a multi-prototype Bradley-Terry encoder and place LLM personas in the same space using a Semantic Similarity Rating (SSR) protocol over stratified probe items. We compare grounded personas (seeded from a real user’s history) and free-form personas against three classical synthetic baselines under a unified Fidelity Gap Index (FGI) over five distributional metrics. Grounding cuts FPD by roughly 30x over free-form prompting yet still leaves only 4.6% to 9.4% PRDC Coverage and intra-group cosine 0.94 to 0.99 against 0.56 to 0.68 for matched real users; item-level metrics look strong (rank accuracy 0.72 to 0.85) and hide this collapse. Local item-level fidelity does not buy global preference coverage, and the gap holds across two open-weights LLMs and four product splits.