Timezone: »

 
Poster
A Closer Look at Smoothness in Domain Adversarial Training
Harsh Rangwani · Sumukh K Aithal · Mayank Mishra · Arihant Jain · Venkatesh Babu Radhakrishnan

Tue Jul 19 03:30 PM -- 05:30 PM (PDT) @ Hall E #526
in Poster Session 1 »

Domain adversarial training has been ubiquitous for achieving invariant representations and is used widely for various domain adaptation tasks. In recent times, methods converging to smooth optima have shown improved generalization for supervised learning tasks like classification. In this work, we analyze the effect of smoothness enhancing formulations on domain adversarial training, the objective of which is a combination of task loss (eg. classification, regression etc.) and adversarial terms. We find that converging to a smooth minima with respect to (w.r.t.) task loss stabilizes the adversarial training leading to better performance on target domain. In contrast to task loss, our analysis shows that converging to smooth minima w.r.t. adversarial loss leads to sub-optimal generalization on the target domain. Based on the analysis, we introduce the Smooth Domain Adversarial Training (SDAT) procedure, which effectively enhances the performance of existing domain adversarial methods for both classification and object detection tasks. Our analysis also provides insight into the extensive usage of SGD over Adam in the community for domain adversarial training.

Keywords: T: Domain Adaptation and Transfer Learning · T: Optimization · MISC: Transfer, Multitask and Meta-learning

Author Information

Harsh Rangwani (Indian Institute of Science)

Phd Student at Video Analytics Lab, Indian Institute of Science. Supported by Prime Minister's Research Fellowship.

Sumukh K Aithal (PES University)
Mayank Mishra (Indian Institute of Science, Bangalore)
Arihant Jain (Indian Institute of Science)
Venkatesh Babu Radhakrishnan (Indian Institute of Science)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors