Representational Geometry Reveals How Context Structures Concept Spaces in Language Models

How context shapes the meaning of a concept is a foundational question in cognitive science and machine learning, yet direct experimental investigation remains difficult. Large language models, trained on vast human-generated text, offer a computational window into the structure of conceptual representation. The dominant view in machine learning treats concept representations as stationary geometric objects. Yet concepts appear in context, and context transforms them. We ask whether this transformation has shared structure and whether that structure is semantically organized in ways that reflect human conceptual knowledge. Drawing from neural population geometry, we formalize concept representations as point-cloud manifolds and contextual transformations as vector fields. Across six model families from 500M to 30B parameters, we investigate natural, artificial, and abstract concepts under six semantic context dimensions grounded in theories of human conceptual representation. We find that context moves each concept differently. The variance in displacement is semantically organized, correlating with lexical concreteness and density. Importantly, this variance structure is shared across models. Displacement structure transferred from one model predicts held-out displacements in other models significantly above chance, and ablating this structure degrades prediction. These findings suggest that the geometry of how context transforms concepts is not a property of any particular model, but a stable structure that may reflect something deeper about how meaning is organized. Collectively, these findings offer computational insight into how concepts can be simultaneously stable and context-sensitive.