Pitfalls and Remedies for Multi-Task Bayesian Optimization

Bayesian optimization routinely warm-starts target experiments with data from related source tasks, and the multi-task Gaussian process is the text- book surrogate for the job. We revisit this default in a controlled setting and find that it misestimates the cross-task correlation even on the simplest non-trivial case: affinely related source and target tasks, where a working transfer-learning method should obviously succeed. We trace the failure to two independent structural mechanisms. Per-task standardization, the textbook fix for the affine slice ambiguity, propagates a finite-sample align- ment error into the recovered correlation. The marginal likelihood itself identifies the correlation only at a per-sample rate that a Gaussian pro- cess at non-overlapping designs further dilutes. We propose three conser- vative remedies that follow from the analysis: promoting per-task means and scales to model parameters, restricting the task covariance to non- negative correlations, and co-locating part of the source and target de- signs. Across a multi-task BO grid and a transfer-learning sweep on an instruction-following benchmark, these remedies recover the vanilla base- line on the simple cells, while the broader failure persists on harder cells and across most rank-based and latent-context variants.