Biophysics-Guided Genomic Foundation Models via Attention Gating

Predicting DNA regulatory function demands models that integrate sequence context with the physical chemistry of nucleic acids. Genomic foundation models (GFMs) are widely used for this task, yet we show they are biophysically blind: attention weights ignore biophysical structure (regulatory attention score ≈ 1.0), representations occupy <4% of embedding dimensionality, and pretraining on raw sequence statistics degrades performance on certain regulatory tasks. We introduce PhysGFM (Physics-Guided Genomic Foundation Models), which injects biophysical knowledge into every attention head via a scalar gate g = σ(W_g · [x; φ(x)]), where φ(x) ∈ ℝ¹¹ encodes stacking free energy, melting temperature, dinucleotide entropy, and minor groove width—computed analytically from sequence alone. PhysGFM adds only 184K parameters (0.19% overhead) with no architectural changes. On the Nucleotide Transformer v2, it improves all 12 GUE tasks at 100M scale (+0.0170 mean MCC, t=4.91, p<0.001, n=36) and 11/12 at 250M (+0.0091, p<0.01). Mechanistic analysis reveals task-adaptive attention calibration: PhysGFM focuses attention for promoters (entropy −0.07) and broadens it for chromatin marks (+0.05 to +0.13), increasing intrinsic dimensionality by +3.2–10.6 across all tasks. Cross-species validation on E. coli σ70 promoters via the Mechanistic Invariance Test yields CSS = 0.74 (p<0.0001), outperforming HyenaDNA (0.63) and Evo2-1B (0.60)—notably, the gate is trained exclusively on human data, demonstrating that biophysical priors transfer across species without retraining. A controlled ablation confirms that 11 curated features outperform a 45-dimensional expansion (+0.0170 vs. +0.0086): targeted physical priors beat breadth of coverage. By recovering regulatory intelligence that sequence-only training leaves on the table at <0.2% parameter cost, PhysGFM establishes biophysical attention gating as a general mechanism for grounding foundation models in the physical sciences.