A Compositional Calculus for Semantic Synergy in Language Model Embeddings

We introduce semantic synergy: a training-free measure of non-compositional representation in language models, obtained by taking the discrete derivative of a phrase embedding over its sub-span structure. Formally, semantic synergy is the Möbius inverse of the embedding function on the partial order of contiguous sub-spans. Across two embedding models and 107 pairs of short English idiomatic and literal phrases, semantic synergy strongly separates idiomatic from literal phrases (Cohen's $d \approx 1.80$--$1.81$, $p < 10^{-28}$), outperforming alternative residuals. The measure further distinguishes non-compositional proper names in a supporting experiment, and yields steering directions that move phrase embeddings toward idiomatic interpretations. Layer-wise extraction in Qwen3-0.6B and Pythia-1B models shows that the non-compositional structure emerges mainly in middle-to-late layers, and becomes strong only late in training. Span-Möbius residuals therefore provide a lightweight algebraic probe of compositional structure in embedding spaces and a bridge toward hidden-state mechanistic analysis.