Representative vs. Load-bearing Layers: A Dissociation in Genomic Foundation Models

Genomic foundation models are typically evaluated by aggregating representations across layers or defaulting to the last layer; neither asks which layer the joint model actually relies on. We probe this with a minimal training-free scalar — the L2 norm of the per-layer hidden-state shift at the variant token, $\|\Delta h_\ell\|_2$ — on $8{,}008$ ClinVar single-nucleotide variants in NT-v2 500M (a masked language model, MLM) and Evo 2 7B (a causal language model, CLM, with a hyena/attention hybrid). In both models, the layer with peak single-feature AUROC (*representative*) is not the layer a joint multi-layer classifier most depends on (*load-bearing*, identified by leave-one-layer-out ablation drop, concordant with $|\mathrm{SHAP}|$). Representative layers sit mid-network in both models, while load-bearing depth lies at opposite ends of the depth axis — mid-shallow in the MLM, deep in the CLM hybrid. The dissociation has direct downstream consequences. In NT-v2, a 1-dim mid-layer scalar exceeds the canonical 1024-dim last-layer mean-pool baseline by $+0.049$ AUROC; in Evo 2, the 4096-dim mean-pool is competitive with the joint $\|\Delta h_\ell\|_2$ feature. Standard last-layer pooling therefore leaves variant-relevant signal untapped specifically in MLM-based pipelines.