Transformed Latent Variable Multi-Output Gaussian Processes

Multi-Output Gaussian Processes (MOGPs) provide a principled probabilistic framework for modelling correlated outputs but face scalability bottlenecks when applied to datasets with high-dimensional output spaces. To maintain tractability, existing methods typically resort to restrictive assumptions, such as employing low-rank or sum-of-separable kernels, which can limit expressiveness. We propose the Transformed Latent Variable MOGP (T-LVMOGP), a novel framework that scales MOGPs to a massive number of outputs while preserving the capacity to capture meaningful inter-output dependencies. T-LVMOGP constructs a flexible multi-output deep kernel by mapping inputs and output-specific latent variables into an embedding space using a Lipschitz-regularised neural network. Combined with stochastic variational inference, our model effectively scales to high-dimensional output settings. Across diverse benchmarks, including climate modelling with over $10,000$ outputs and zero-inflated spatial transcriptomics data, T-LVMOGP outperforms baselines in both predictive accuracy and computational efficiency.

Many real-world problems require predicting many related quantities at the same time. For example, climate models may need to estimate environmental conditions at thousands of locations, while biological studies may measure thousands of genes across tissue regions. These quantities are usually connected, so treating them separately can miss important patterns. This work develops a new method for making such predictions more accurately and efficiently. The main idea is to give each predicted quantity a compact description of its own characteristics, then use these descriptions to learn how different quantities are related. This allows the model to share useful information across outputs while still recognising that each output may behave differently. A key challenge is that modelling a large-scale of related quantities can be extremely slow and memory-intensive. Our proposed method is designed to avoid this problem by using an efficient approximation strategy, making it practical for much larger datasets than standard approaches.