Noisy labels are ubiquitous in real-world datasets, which poses a challenge for robustly training deep neural networks (DNNs) as DNNs usually have the high capacity to memorize the noisy labels. In this paper, we find that the test accuracy can be quantitatively characterized in terms of the noise ratio in datasets. In particular, the test accuracy is a quadratic function of the noise ratio in the case of symmetric noise, which explains the experimental findings previously published. Based on our analysis, we apply cross-validation to randomly split noisy datasets, which identifies most samples that have correct labels. Then we adopt the Co-teaching strategy which takes full advantage of the identified samples to train DNNs robustly against noisy labels. Compared with extensive state-of-the-art methods, our strategy consistently improves the generalization performance of DNNs under both synthetic and real-world training noise.
Pengfei Chen (The Chinese University of Hong Kong)
Ben Liao (Tencent)
PhD in Mathematics, Paris, France. Research interests include, machine learning theory, visual understanding, graph neural networks, geometric deep learning.
Guangyong Chen (Tencent)
Shengyu Zhang (Tencent; The Chinese University of Hong Kong)
Related Events (a corresponding poster, oral, or spotlight)
2019 Oral: Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels »
Wed Jun 12th 06:30 -- 06:35 PM Room Room 102