Where Simple Baselines Fail: Mapping the Modeling Frontier of Perturbation Prediction

Whether deep learning models for perturbation prediction outperform simple baselines remains contested. We argue this debate is ill-posed: standard benchmarks mix perturbations already well-predicted by simple rules with others whose reproducible signal remains unexplained. We introduce Baseline Saturation, a per-perturbation measure of how much of the control-bounded dynamic range a matched simple baseline already captures. Across nine CRISPR perturbation datasets and six generalization regimes, we find that within-regime heterogeneity dominates: even within a single scenario, perturbation-level saturation spans the full range, with approximately 40\% of evaluations near-solved by baselines and 30\% remaining resistant. Deep learning models (scGPT, GEARS, PRESAGE) outperform baselines specifically on resistant perturbations, recovering which genes respond and in which direction, but not response magnitudes. Because saturated perturbations dominate most test sets, these gains are masked by aggregate reporting. Biologically, resistance is driven by the uniqueness of a perturbation's response relative to the dataset mean and is cell-type-dependent rather than gene-intrinsic. Our results reframe evaluation: simple baselines define the solved regime, and their failures map the frontier where expressive models add value.