Timezone: »

Automated Synthetic-to-Real Generalization
Wuyang Chen · Zhiding Yu · Zhangyang Wang · Anima Anandkumar

Wed Jul 15 04:00 PM -- 04:45 PM & Thu Jul 16 04:00 AM -- 04:45 AM (PDT) @ None #None

Models trained on synthetic images often face degraded generalization to real data. As a convention, these models are often initialized with ImageNet pretrained representation. Yet the role of ImageNet knowledge is seldom discussed despite common practices that leverage this knowledge to maintain the generalization ability. An example is the careful hand-tuning of early stopping and layer-wise learning rates, which is shown to improve synthetic-to-real generalization but is also laborious and heuristic. In this work, we explicitly encourage the synthetically trained model to maintain similar representations with the ImageNet pretrained model, and propose a \textit{learning-to-optimize (L2O)} strategy to automate the selection of layer-wise learning rates. We demonstrate that the proposed framework can significantly improve the synthetic-to-real generalization performance without seeing and training on real data, while also benefiting downstream tasks such as domain adaptation. Code is available at: https://github.com/NVlabs/ASG.

Author Information

Wuyang Chen (Texas A&M University)
Zhiding Yu (NVIDIA)

Zhiding Yu is a Senior Research Scientist at NVIDIA. Before joining NVIDIA in 2018, he received Ph.D. in ECE from Carnegie Mellon University in 2017, and M.Phil. in ECE from The Hong Kong University of Science and Technology in 2012. His research interests mainly focus on deep representation learning, weakly/self-supervised learning, transfer learning and deep structured prediction, with their applications to vision and robotics problems.

Zhangyang Wang (University of Texas at Austin)
Anima Anandkumar (Amazon AI & Caltech)

More from the Same Authors