Assistive Prompt Mediation: Evaluating Language Models Under Accessibility Constraints

Large language models (LLMs) are increasingly used as assistive interfaces for users who cannot reliably produce clean text due to accessibility constraints, yet existing evaluations assume iterative input repair and focus on task accuracy or generic noise robustness. We introduce Assistive Prompt Mediation (APM), a theory-grounded evaluation paradigm that reframes assistance as a constrained mediation problem: recovering latent user intent from accessibility-impaired input without clarification, while minimizing cognitive burden and hallucination risk. APM decomposes assistive quality along these axes and is instantiated across 8 languages, 4 accessibility-driven noise classes, and 10 frontier LLMs, with impairment severity yielding accessibility sensitivity curves. Results show that apparent robustness often masks trade-offs—high intent preservation frequently coincides with increased burden or hallucinated mediation, hallucination rates vary by more than $2\times$ across noise types, and assistive decisions exhibit bounded entropy ($<0.81$ normalized), indicating systematic rather than unstable behavior. These findings demonstrate that standard robustness metrics substantially overestimate assistive reliability and motivate evaluating LLMs as constrained mediators under accessibility-driven input degradation.

Many people use AI chat systems, called large language models, as writing or communication assistants. But people who use accessibility tools, or who experience dyslexia, motor impairments, speech-recognition errors, low vision, or fatigue, may send messages that are misspelled, shortened, or incomplete. Today’s evaluations often ask whether models can still answer noisy prompts, but that misses a key question: can the model safely rewrite the message into what the user meant without asking them to do more work? We propose Assistive Prompt Mediation, a way to test language models in this setting. It checks whether a model preserves the user’s intent, reduces the effort needed to read or fix the prompt, and avoids adding unsupported details. We tested ten language models on more than 40,000 accessibility-affected prompts across eight languages and four kinds of realistic input difficulty. The results show that models can look reliable while still creating problems: they often keep the main intent but make the rewritten prompt longer, more complex, or filled with assumptions. This matters because assistive AI should reduce barriers, not shift extra work back to the people it is meant to support.