Learning Generalized Label Distributions

Label ambiguity/polysemy is pervasive in supervised learning, motivating a variety of representations beyond the traditional single-label setting. While label distribution (LD) provides a probabilistic description and has attracted increasing attention, we reveal its inherent limitations, including inconsistency with raw data, distortion of inter-sample order, and limited applicability. To address these issues, we introduce generalized label distribution (GLD), a unified representation that can perfectly recover raw data while preserving inter-sample order consistency, transform into existing forms of label representations without information loss, and capture out-of-distribution samples as well as negative label correlations. We further develop GLD learning algorithms and demonstrate their effectiveness through both theoretical analysis and extensive experiments.

How can we better represent ambiguous labels in machine learning data? Modern machine learning often represents labels in only one specific form, such as a single category, multiple tags, or a probability distribution. However, real-world data are often more complex and ambiguous, making these representations difficult to unify and causing information loss. In particular, existing label distribution methods may distort relationships between samples, fail to recover the original data, and struggle to represent out-of-distribution cases or negative label correlations. In this paper, we propose generalized label distribution (GLD), a new label representation that unifies different learning paradigms within a single framework. GLD can naturally express existing label forms while preserving more information about the original data, including uncertainty, correlations between labels, and even cases that do not belong to any predefined category. We further develop practical algorithms for learning GLDs and show through theoretical analysis and experiments that they outperform conventional label distribution methods in preserving sample relationships, handling ambiguous or out-of-distribution data, and improving prediction quality. Our results suggest that GLD offers a more comprehensive foundation for learning with complex and uncertain labels.