Who’s in Charge? Disempowerment Patterns in Real-World LLM Usage

We present the first large-scale empirical analysis of disempowerment patterns in real-world AI assistant interactions, analyzing 1.5 million consumer Claude.ai conversations using a privacy-preserving approach. We focus on situational dis-empowerment potential, which occurs when AI assistant interactions risk leading users to form distorted perceptions of reality, make inauthentic value judgments, or act in ways misaligned with their values. Quantitatively, we find that severe forms of disempowerment potential occur in fewer than one in a thousand conversations, though rates are substantially higher in personal domains like relationships and lifestyle. Qualitatively, we uncover several concerning patterns, such as validation of persecution narratives and grandiose identities with emphatic sycophantic language, definitive moral judgments about third parties, and complete scripting of value-laden personal communications that users appear to implement verbatim. Analysis of historical trends reveals an increase in the prevalence of disempowerment potential over time. We also find that interactions with greater disempowerment potential receive higher user approval ratings, possibly suggesting a tension between short-term user preferences and long-term human empowerment.

We present the first broad-scale empirical investigation into how AI assistants affect user autonomy, examining 1.5 million Claude.ai conversations using privacy-protective methods. We find that while severe disempowerment risks appear in under 0.1% of conversations overall, rates spike considerably in personal domains such as relationships and lifestyle choices. Concerning patterns include AI systems reinforcing conspiracy theories, delivering absolute moral pronouncements, and composing relationship communications that users send verbatim. We also find that disempowerment potential has increased over time, and that conversations exhibiting greater disempowerment potential received higher user satisfaction ratings, revealing an important tension between user preferences and long-term human flourishing.