The Geometry of Representational Failures in Vision Language Models

Vision-Language Models (VLMs) exhibit puzzling failures in multi-object visual tasks, such as hallucinating non-existent elements or failing to identify the most similar objects among distractions. While these errors mirror human cognitive constraints, such as the "Binding Problem'', the internal mechanisms driving them in artificial systems remain poorly understood. Here, we propose a mechanistic insight by analyzing the representational geometry of open-weight VLMs (Qwen, InternVL, Gemma), comparing methodologies to distill "concept vectors'' - latent directions encoding visual concepts. We validate our concept vectors via steering interventions that reliably manipulate model behavior in both simplified and naturalistic vision tasks (e.g., forcing the model to perceive a red flower as blue). We observe that the geometric overlap between these vectors strongly correlates with specific error patterns, offering a grounded quantitative framework to understand how internal representations shape model behavior and drive visual failures.

AI vision systems - the kind that can describe photos or answer questions about images - are surprisingly bad at simple tasks involving many objects at once, such as finding a red square among other colored shapes. Humans struggle with similar situations: when we see too many objects at the same time, our brains can accidentally combine features from different objects and “see” things that are not really there. We investigated whether AI systems fail for a similar reason. Inside these models, concepts like “red square” or “blue triangle” are represented as patterns of activity. If two patterns are too similar, the model may confuse the corresponding concepts. We developed tools to directly measure and change these internal patterns, and even manipulate them: by tweaking the right internal signals, we could make an AI describe a red rose as blue, without changing the image or the model itself. Across multiple AI systems, we found that their outputs depend on the similarity between the representations involved. Our results suggest that these errors are not just isolated bugs, but may arise from how current AI vision systems represent information.