Timezone: »
Recent deep networks are capable of memorizing the entire data even when the labels are completely random. To overcome the overfitting on corrupted labels, we propose a novel technique of learning another neural network, called MentorNet, to supervise the training of the base deep networks, namely, StudentNet. During training, MentorNet provides a curriculum (sample weighting scheme) for StudentNet to focus on the sample the label of which is probably correct. Unlike the existing curriculum that is usually predefined by human experts, MentorNet learns a data-driven curriculum dynamically with StudentNet. Experimental results demonstrate that our approach can significantly improve the generalization performance of deep networks trained on corrupted training data. Notably, to the best of our knowledge, we achieve the best-published result on WebVision, a large benchmark containing 2.2 million images of real-world noisy labels.
Author Information
Lu Jiang (Google)
Lu Jiang is a senior research scientist at Google Research and an adjunct faculty at Carnegie Mellon University. He received a Ph.D. in Artificial Intelligence (Language Technology) from Carnegie Mellon University in 2017. Lu's primary interests lie in the interdisciplinary field of Multimedia, Machine Learning, and Computer Vision. He is the recipient of Yahoo Fellow and Erasmus Mundus Scholar. He received Best paper nomination at ACL and ICMR; Best poster at IEEE SLT; Best system in NIST TRECVID. He was a key contributor to IARPA Aladdin project, and helped create YouTube-8M dataset and AutoML in Google. He served as an area chair for ACM Multimedia, AVSS; the PC for premier conferences such as CVPR, ICML, ICCV, AAAI and IJCAI; the reviewer for NSF SBIR/STTR Panel, Google Faculty Award, and Google Research Hiring Committee; the journal reviewer for JMLR, TPAMI, JAIR, TMM, CVIU, etc.
Zhengyuan Zhou (Stanford University)
Thomas Leung (Google Inc)
Li-Jia Li (Google)
Li Fei-Fei (Stanford University & Google)
Related Events (a corresponding poster, oral, or spotlight)
-
2018 Oral: MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels »
Thu. Jul 12th 12:50 -- 01:00 PM Room A6
More from the Same Authors
-
2023 Poster: Muse: Text-To-Image Generation via Masked Generative Transformers »
Huiwen Chang · Han Zhang · Jarred Barber · Aaron Maschinot · Jose Lezama · Lu Jiang · Ming-Hsuan Yang · Kevin Murphy · William Freeman · Michael Rubinstein · Yuanzhen Li · Dilip Krishnan -
2021 Poster: Self-supervised and Supervised Joint Training for Resource-rich Machine Translation »
Yong Cheng · Wei Wang · Lu Jiang · Wolfgang Macherey -
2021 Spotlight: Self-supervised and Supervised Joint Training for Resource-rich Machine Translation »
Yong Cheng · Wei Wang · Lu Jiang · Wolfgang Macherey -
2020 Poster: Gradient-free Online Learning in Continuous Games with Delayed Rewards »
Amélie Héliou · Panayotis Mertikopoulos · Zhengyuan Zhou -
2020 Poster: Beyond Synthetic Noise: Deep Learning on Controlled Noisy Labels »
Lu Jiang · Di Huang · Mason Liu · Weilong Yang -
2020 Poster: Finite-Time Last-Iterate Convergence for Multi-Agent Learning in Games »
Tianyi Lin · Zhengyuan Zhou · Panayotis Mertikopoulos · Michael Jordan -
2020 Poster: Distributionally Robust Policy Evaluation and Learning in Offline Contextual Bandits »
Nian Si · Fan Zhang · Zhengyuan Zhou · Jose Blanchet -
2018 Poster: Distributed Asynchronous Optimization with Unbounded Delays: How Slow Can You Go? »
Zhengyuan Zhou · Panayotis Mertikopoulos · Nicholas Bambos · Peter Glynn · Yinyu Ye · Li-Jia Li · Li Fei-Fei -
2018 Oral: Distributed Asynchronous Optimization with Unbounded Delays: How Slow Can You Go? »
Zhengyuan Zhou · Panayotis Mertikopoulos · Nicholas Bambos · Peter Glynn · Yinyu Ye · Li-Jia Li · Li Fei-Fei