Metric Choice Determines Semantic Geometry in LLM Hidden States: A measurement Study of Hidden-State Geometry

Large language models (LLMs) can be viewed as functions mapping discrete prompts to continuous hidden representations. Recent work shows that decoder-only transformers are almost surely injective, implying that distinct prompts are not collapsed into identical hidden states. Building on this functional perspective, we study whether LLM representations also exhibit expansion: the layerwise growth of distances between prompt representations. We empirically measure pairwise hidden-state distances across transformer depth and find that raw Euclidean (L2) distances generally increase with depth, indicating broad expansion in representation space. Although expansion appears across meaningful, unrelated, and random-token prompt families, these families are not geometrically identical. Raw L2 distance primarily reflects shared norm growth across layers, making expansion appear largely independent of semantic meaning or lexical coherence. However, when representations are compared using centered cosine distance, semantic and topic-level distinctions become visible. These results separate two geometric phenomena in LLMs: intrinsic expansion, which appears to be a general property of hidden-state evolution, and semantic geometry, which becomes visible only under appropriate measurement protocols.