Token-Wise Residual Latent Adapters: Steering Seq2Seq Models for Protein Fitness Extrapolation

Protein design requires extrapolating beyond training data to achieve higher fitness. State-of-the-art methods typically fine-tune billion-parameter language models end-to-end, often combined with external scorers, data distillation, and multiple rounds of iterative refinement. We introduce a residual latent adapter, a 5M parameter MLP inserted between the encoder and decoder of a frozen ProtT5-3B model, which learns a token-wise residual transformation on encoder embeddings via a simple MSE objective. In a single forward pass with no external scorer, RLA achieves comparable or superior fitness to methods requiring 600$\times$ more trainable parameters and multi-stage pipelines, particularly on the harder extrapolation benchmarks most relevant to practical protein engineering. Our results demonstrate that a compact residual transformation in latent space provides a simple, data-efficient, and compute-efficient approach to protein fitness extrapolation.