Convex Dataset Valuation for Post-Training

Improving LLM performance on downstream tasks sometimes requires leveraging auxiliary datasets during post-training. In practice, however, developers face constraints on compute, labeling, and licensing costs that preclude using all available data, necessitating principled dataset-level selection. These constraints are increasingly shaped by dataset marketplaces, where data acquisition is governed by budgets and negotiation. We study dataset valuation as a subset selection problem during LLM post-training. Our goal is to identify and weight auxiliary datasets so as to maximize target task performance given constrained budgets. We first show that commonly used gradient alignment scores provide a reasonable yet incomplete valuation signal, as they ignore redundancy among datasets. To address this, we propose a scalable convex dataset-level valuation method based on kernel mean matching (KMM) in gradient space, which jointly accounts for alignment with the target task and redundancy across auxiliary datasets. Through extensive experiments across diverse post-training settings and tasks, we show that our approach consistently outperforms existing valuation baselines, achieving stronger performance with low computational overhead. Our results position dataset valuation as a practical decision tool for post-training data selection in market-constrained large language model settings.

Large language models can get better at specific jobs when they learn from extra data, but not all data is equally useful. In the real world, teams may have to pay for additional annotation or computing time, so they cannot simply use everything they find. Then given a limited budget, how should a team decide which datasets are worth using? A common way to choose data is to look for datasets that seem most relevant to the target task, but several datasets may teach the model the same thing, so paying for all of them adds little value. We developed a method that looks not only at how useful each dataset is, but also at how much it overlaps with the others, so the final selection is both helpful and cost-effective. Our method is efficient enough to use during large language model post-training, and in experiments across several tasks, it chose better combinations of datasets than existing approaches.