Timezone: »

 
To Aggregate or Not? Learning with Separate Noisy Labels
Jiaheng Wei · Zhaowei Zhu · Tianyi Luo · Ehsan Amid · Abhishek Kumar · Yang Liu

@
in DMLR Workshop: Data-centric Machine Learning Research »

The rawly collected training data often comes with separate noisy labels collected from multiple imperfect annotators (e.g., via crowdsourcing). A typical way of using these separate labels is to first aggregate them into one and apply standard training methods. The literature has also studied extensively on effective aggregation approaches. This paper revisits this choice and aims to provide an answer to the question of whether one should aggregate separate noisy labels into single ones or use them separately as given. We theoretically analyze the performance of both approaches under the empirical risk minimization framework for a number of popular loss functions, including the ones designed specifically for the problem of learning with noisy labels. Our theorems conclude that label separation is preferred over label aggregation when the noise rates are high, or the number of labelers/annotations is insufficient. Extensive empirical results validate our conclusions.

Author Information

Jiaheng Wei (University of California, Santa Cruz)
Zhaowei Zhu (University of California, Santa Cruz)
Tianyi Luo (Amazon)
Ehsan Amid (Google DeepMind)
Abhishek Kumar (Google Brain)
Yang Liu (UC Santa Cruz/ByteDance Research)

More from the Same Authors