WARP: Weight-Space Analysis for Recovering Training Data Portfolios

Foundation models are routinely released to the public, yet the data recipes used to train them---such as the domain mixtures that determine how different sources are sampled---are rarely disclosed. This creates an access asymmetry: researchers can study the resulting models but lack visibility into the training distributions that produced them. Prior works for inferring training data, such as membership inference, detect at the level of individual samples and therefore cannot characterize the global composition of a model's training corpus. We introduce WARP, a framework that recovers a fine-tuned model's domain mixture directly from its released weights. WARP interpolates between the base and fine-tuned models using model merging, generating \emph{pseudo-checkpoints} that approximate the missing training trajectory and expose a geometric footprint of the training data in weight space. From these footprints, WARP extracts geometric features and maps them to domain proportions using either a parameter-free softmax readout or a MLP projector trained on synthetic mixtures. In controlled experiments with BERT and GPT-2, WARP recovers domain mixtures with MAE as low as $0.048$ and $0.117$ respectively, outperforming membership inference and a variant with access to the true training trajectory, and remains accurate when recovering different training recipes.