Evolutionary Curriculum Learning for Biological Sequence Modeling

Variational autoencoders (VAEs) trained on multiple sequence alignments (MSAs) have emerged as powerful generative models for biological sequences, with applications ranging from disease variant prediction to functional RNA design. However, standard biological VAE training treats all sequences as exchangeable, ignoring the rich evolutionary structure that organizes homologous sequences from evolutionarily close to highly divergent. We propose Evolutionary Curriculum Learning (ECL), a training strategy that exploits this structure by progressively exposing the model to sequences of increasing evolutionary distance from sampled anchors, following a power-law expansion schedule. Applied to two architecturally distinct VAE models and two biological domains—protein variant effect prediction with EVE and RNA family sequence generation with RfamGen—ECL consistently improves downstream task performance: ClinVar variant classification AUROC increases from 0.842 to 1.000 for PTEN and from 0.972 to 0.986 for p53, while mean covariance model bit scores for generated RNA sequences improve by 1.9-7.1% across three diverse RNA families. These results suggest that evolutionary distance is a broadly effective inductive bias for ordering the training curriculum for biological sequence modeling.