Nested birth-death processes are competitive with neural networks as time-dependent models of protein evolution

Most statistical phylogenetics analyses use simple continuous-time finite-state Markov models of point substitution to describe molecular evolution. These models enforce unrealistic assumptions like keeping sequence length fixed, ignoring insertions and deletions (indels) entirely, and making little (if any) allowance for variation in selection pressure due to interactions between amino acids. We extend the TKF92 model—the canonical hierarchical model combining an outer birth-death process for indels with an inner finite-state Markov chain for substitutions—by introducing additional nesting and latent states, allowing for structural heterogeneity. We compare these TKF92 extensions to two classes of neural seq2seq models that use evolutionary time as an input feature: the first "basic" class lacks any evolutionary modeling constraints, while the second "hybrid" class combines neural sequence embeddings with a TKF92-like likelihood function. We evaluate the per-character perplexities of all models on splits of the Pfam database of aligned protein domains. The hybrid neural models outperform their basic counterparts across all sequence embedding architectures. Furthermore, a nested TKF-based model with only 30,000 parameters is highly competitive with all neural networks (which contain tens of millions of parameters), outperforming all but two of the neural architectures tested. Taken together, our results indicate that approaches grounded in molecular evolutionary theory may provide a better fit to real alignments than unconstrained alternatives, supporting the incorporation of CTMC-based model structure within future neural phylogenetic approaches.

Proteins mutate over millions of years. Modeling how they change helps scientists understand diseases, design drugs, and reconstruct evolutionary history. Existing mathematical models of protein evolution are fast and interpretable but too simple to capture biological complexity, while modern neural networks are powerful but are difficult to interpret and often ignore evolutionary time. We asked how we could make more powerful time-dependent models of protein evolution. We extended a classical model of molecular evolution that describes how proteins gain, lose, and alter amino acids over time by adding mixtures and hierarchical structure. This allows different regions of a protein to evolve at different rates, and those rates can depend on surrounding amino acids (which is biologically intuitive). We also built neural networks that predict site-specific evolutionary rates, blending deep learning with existing evolutionary frameworks. All models were evaluated on over one million aligned protein pairs from Pfam, a well-established database of protein families. Neural networks that incorporated evolutionary structure consistently outperformed those that did not, showing that encoding classical frameworks into neural architectures improves model fit. Our most elaborate hierarchical model — with three orders of magnitude fewer parameters — matched or outperformed other neural networks. These results suggest that classical models of molecular evolution still remain important for describing the dynamics of biological sequences, even in the age of deep learning.