Timezone: »

 
Spotlight
Disentangling Sampling and Labeling Bias for Learning in Large-output Spaces
Ankit Singh Rawat · Aditya Menon · Wittawat Jitkrittum · Sadeep Jayasumana · Felix Xinnan Yu · Sashank Jakkam Reddi · Sanjiv Kumar

Wed Jul 21 05:25 AM -- 05:30 AM (PDT) @
in Supervised Learning 1 »

Negative sampling schemes enable efficient training given a large number of classes, by offering a means to approximate a computationally expensive loss function that takes all labels into account. In this paper, we present a new connection between these schemes and loss modification techniques for countering label imbalance. We show that different negative sampling schemes implicitly trade-off performance on dominant versus rare labels. Further, we provide a unified means to explicitly tackle both sampling bias, arising from working with a subset of all labels, and labeling bias, which is inherent to the data due to label imbalance. We empirically verify our findings on long-tail classification and retrieval benchmarks.

Keywords: Supervised Learning · Algorithms

Author Information

Ankit Singh Rawat (Google)
Aditya Menon (Google Research)
Wittawat Jitkrittum (Google Research)
Sadeep Jayasumana (Google Research)
Felix Xinnan Yu (Google)
Sashank Jakkam Reddi (Google)
Sanjiv Kumar (Google Research, NY)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors