BioToken and BioFM – Biologically-Informed Tokenization Enables Accurate and Efficient Genomic Foundation Models

Existing genomic foundation models (GFMs) typically treat DNA as raw nucleotide sequences, often overlooking the regulatory context required to interpret genetic variation accurately. We introduce BioToken, a tokenization framework that directly encodes variants and biological annotations into genomic representations, and BioFM, a parameter-efficient model built on this architecture. By leveraging biological inductive biases, BioFM outperforms state-of-the-art models and specialized baselines like Enformer on benchmarks including pathogenicity and expression prediction while requiring 100-fold less compute than current large-scale genomic models. These findings demonstrate that explicitly modeling biological structure yields more robust and efficient genomic representations than scaling alone.

DNA is the instruction manual for life, but small typos in this manual can cause diseases. Recent AI models try to read DNA just like language models read text, looking primarily at the raw sequence of letters. Unfortunately, massive amounts of compute are required to teach these AI models even basic biology of DNA and DNA mutations. To solve this, we created BioToken, a new way to preprocess DNA that annotates the raw genetic code with location and mutation tokens. Using this framework, we built BioFM, a highly efficient AI model. We told the model from the start that some DNA sequences encode proteins, and some mutations are more likely to occur. As a result, BioFM accurately predicts how genetic changes affect health and everything that happens inside our cells. It outperforms massive, top-tier models while using 100 times less compute. Ultimately, this work shows that giving AI a biological head start is far smarter and more practical than simply scaling up model size.