Expectation Alignment of Language Models for Real-World User Expectations

Large language models (LLMs) have demonstrated remarkable performance on standard benchmarks, yet it remains largely unexplored whether they truly meet user expectations. Existing evaluation approaches, relying on model heuristics, expert rubrics, or user simulation, fail to capture the diversity and subtlety of real human expectations, causing models to appear competent while misaligning with what users actually seek. we present the first systematic study of user expectations in real-world LLM interactions, proposing a principled procedure to extract semantically rich expectations and introducing ExpectBench, a benchmark grounded in real user expectations. Analyses reveal that current LLMs struggle to satisfy and anticipate what users hope to obtain, highlighting a fundamental source of misalignment. Building on these observations, we propose LENS, a lightweight latent expectation–aware response generation framework. LENS enables models to internalize user expectations and generate better-aligned responses, consistently improving expectation satisfaction and underscoring the importance of explicitly modeling user expectations for realistic human–AI alignment.

Why does an AI answer that looks correct still feel wrong? The missing piece is often not task competence, but expectation awareness: the ability to infer what would make the answer useful, appropriate, and satisfactory for this particular user. Users evaluate answers against expectations shaped by their goals, context, constraints, and intended use, many of which remain implicit in the initial request. This paper studies this gap between “task completed” and “what the user actually wanted.” We build EXPECTBENCH from real multi-turn human–AI conversations, using users’ follow-up messages to identify the expectations that earlier responses did not fully satisfy. This allows us to evaluate language models not only by whether their answers look reasonable, but by whether they meet the criteria that real users reveal through interaction. Our results show that current models often miss these expectations and struggle to predict them in advance. We also introduce LENS, a lightweight method that helps a model use hidden signals about user expectations when generating responses. Together, these results suggest that future AI systems should learn not only to complete tasks, but also to understand what people are trying to get from those tasks.