GDTR: Layer-wise Settling Depth Reveals Biological Grammar in Genomic Foundation Models
Yoonjin Cho ⋅ Jiheon Kang ⋅ Subin Park ⋅ Sangwoo Kim
Abstract
Genomic foundation models encode rich sequence regularities, yet existing tools rarely answer *where* in the layer stack a specific biological grammar becomes stable. We introduce GDTR (*Genomic Deep-Thinking Ratio*), a training-free residual-stream lens that assigns each nucleotide token a *settling depth* $c(t)$: the first layer at which its representation stabilises against the post-final-norm reference. On Evo 2 7B, splice donor/acceptor sites settle $\sim 2$ layers earlier than intronic and coding contexts (Cohen's $d=-0.43$), with ENCODE enhancer-like cCREs showing a smaller but measurable shift ($d=-0.19$); a single chr22 calibration transfers to held-out chr17 ($94.6\%$ effect retention). Crucially, $c(t)$ is informative in *both directions*: editing the central GT donor motif deepens settling, whereas shuffling the flanking grammar lets the isolated motif settle $3.18$ layers earlier — so motif edits and flank shuffles dissociate motif detection from grammar integration. Six ClinVar molecular-consequence classes also differ in the layer at which variant-induced residual disruption peaks (Kruskal–Wallis $p=3.0\times10^{-10}$), with synonymous substitutions peaking at the deepest layers. GDTR positions settling depth as a layer-wise interpretability axis for genomic foundation models, complementing existing variant scorers.
Video
Chat is not available.
Successful Page Load