Timezone: »

 
Poster
Mitigating Memorization of Noisy Labels by Clipping the Model Prediction
Hongxin Wei · HUIPING ZHUANG · RENCHUNZI XIE · Lei Feng · Gang Niu · Bo An · Sharon Li

Thu Jul 27 04:30 PM -- 06:00 PM (PDT) @ Exhibit Hall 1 #803
in Poster Session 6 »

In the presence of noisy labels, designing robust loss functions is critical for securing the generalization performance of deep neural networks. Cross Entropy (CE) loss has been shown to be not robust to noisy labels due to its unboundedness. To alleviate this issue, existing works typically design specialized robust losses with the symmetric condition, which usually lead to the underfitting issue. In this paper, our key idea is to induce a loss bound at the logit level, thus universally enhancing the noise robustness of existing losses. Specifically, we propose logit clipping (LogitClip), which clamps the norm of the logit vector to ensure that it is upper bounded by a constant. In this manner, CE loss equipped with our LogitClip method is effectively bounded, mitigating the overfitting to examples with noisy labels. Moreover, we present theoretical analyses to certify the noise-tolerant ability of LogitClip. Extensive experiments show that LogitClip not only significantly improves the noise robustness of CE loss, but also broadly enhances the generalization performance of popular robust losses.

[ PDF] 

Author Information

Hongxin Wei (Nanyang Technological University)
HUIPING ZHUANG (South China University of Technology)
RENCHUNZI XIE (Nanyang Technological University)
Lei Feng (Nanyang Technological University, Singapore)
Gang Niu (RIKEN)
Gang Niu

Gang Niu is currently an indefinite-term senior research scientist at RIKEN Center for Advanced Intelligence Project.

Bo An (Nanyang Technological University)
Sharon Li (University of Wisconsin-Madison)

More from the Same Authors