Monotonic Variational Gaussian Process for Efficient Data Collection

Modeling the learning curve is critical for cost-effective data collection in deep learning systems. Most prior approaches assume a specific parametric learning curve, but these can be inappropriate when no reliable parametric form can be assumed for the learning curve. While Gaussian processes offer flexible nonparametric modeling, existing GP approaches that enforce monotonicity typically introduce intractable factors or require derivative observations. To address this, we propose a Monotonic Variational Gaussian Process for Efficient Data Collection (MOVE), which (i) introduces a novel monotonic variational GP formulation with virtual-derivative factors to enable tractable posterior inference, and (ii) develops an expected shortfall based objective for target-driven data collection. Furthermore, our theoretical analysis shows that expected shortfall provides non-vanishing gradient signals that enable reliable gradient-based optimization. Extensive experiments on classification, segmentation, and detection benchmarks demonstrate consistent improvements over the prior method.

When a company wants to build an AI system that can recognize problems on a factory floor, someone has to first go through a large collection of real world examples and carefully label each one. This process takes time and money, so the company must figure out how many labeled examples from each site are truly enough. Most current methods lean on a simple rule of thumb. They assume that gathering more data always improves the AI by a steady, predictable amount. But the real world is more complicated. specially when data flows in from many different sources that each behave in their own way. We built MOVE, a planning tool that watches how the AI improves as data grows and learns this pattern on its own, without forcing it into a fixed rule. When the AI still falls short, MOVE measures how far it is from the goal rather than only whether it will get there. This helps companies gather just enough data and avoid wasteful effort.