Stochastic Order Learning: An Approach to Rank Estimation Using Noisy Data

Rank estimation under label noise poses a fundamental challenge, as ordinal annotations often exhibit structured uncertainty rather than simple label corruption. In this paper, we reformulate rank estimation with noisy ordinal labels as a stochastic ordering problem, in which each instance is inherently associated with multiple plausible ranks instead of a single deterministic label. Based on this view, we propose stochastic order learning (SOL), a learning framework that captures ordinal label uncertainty and learns an embedding space through two complementary objectives: a discriminative loss that structures instance-centroid interactions and a stochastic order loss that enforces probabilistic ordering relations between instances. Extensive experiments across diverse datasets demonstrate that SOL enables reliable rank estimation under various types and levels of label noise. The source code is available at https://github.com/cwlee00/SOL.

Many AI systems need to predict ordered values, such as a person’s age, an aesthetic score, a medical assessment score, or the quality of a translated sentence. In practice, the training labels for these tasks are often noisy because adjacent ranks can be hard to distinguish, and human judgments may be subjective. Unlike ordinary classification, not all mistakes are equally harmful: predicting a 24-year-old as 26 is much less serious than predicting a 43-year-old as 59. We propose Stochastic Order Learning, or SOL, a method for learning from such noisy ordered labels. Instead of assuming that each training example has one fixed correct rank, SOL treats several nearby ranks as plausible with different probabilities. It then learns an embedding space where examples and rank prototypes are arranged according to these uncertain but ordered relationships. SOL also includes a mechanism to detect and refine highly noisy labels during training. Experiments on age estimation, image aesthetics, medical assessment, and text regression show that SOL makes more reliable predictions under different types and levels of label noise. This can help build AI systems that remain robust when perfect annotations are difficult or expensive to obtain.