LDARNet: DNA Adaptive Representation Network with Learnable Tokenization for Genomic Modeling

Genomic foundation models increasingly adopt large language model architectures, yet almost universally rely on fixed tokenization schemes such as $k$-mers, BPE, or single nucleotides, which impose arbitrary sequence boundaries that may obscure biologically relevant structure. We present LDARNet, a 120M-parameter hierarchical genomic foundation model that adapts H-Net-style dynamic chunking from autoregressive generation to masked language modeling, combining BiMamba-2 state-space layers with local attention, bidirectional routing, and a ratio-based regularizer to induce adaptive token boundaries without supervision. Fine-tuned on 27 tasks from the Nucleotide Transformer and Genomic Benchmarks suites, LDARNet achieves 11/18 wins among compact models ($<$300M parameters) and state-of-the-art results on 5 histone modification tasks, outperforming models up to 20$\times$ larger. A FLOPs-matched controlled experiment isolates learned routing as the source of these gains: learned boundaries beat fixed-grid boundaries by up to 14 percentage points on histone tasks at identical compute. Nucleotide-resolution analysis further shows that the learned boundaries align with canonical promoter motifs and splice junctions without supervision, providing a biological interpretation for adaptive tokenization in genomic foundation models.

DNA is the instruction manual of life, and computers can now read it the way they read text - but unlike English, DNA has no spaces telling the computer where one "word" ends and the next begins. Most genomic AI models work around this by chopping DNA into fixed-length pieces, like cutting a book into 6-letter chunks regardless of where words actually appear. This can split meaningful biological signals across pieces and obscure how cells read their own genome. We built LDARNet, an AI model that learns where to place the cuts instead of using a fixed rule. As it reads DNA during training, it figures out for itself which positions look like natural boundaries. The result is twofold. First, our model matches or beats models up to 20 times its size on a wide range of genomic prediction tasks, especially those involving gene regulation. Second, when we examine where the model decides to cut, those positions line up with real biological landmarks - promoters and splice sites that biologists have known about for decades - even though we never told the model about them. This makes the model both more efficient and more interpretable for biologists.