Timezone: »

 
Towards Principled Disentanglement for Domain Generalization
Hanlin Zhang · Yi-Fan Zhang · Weiyang Liu · Adrian Weller · Bernhard Schölkopf · Eric Xing

@
in Machine Learning for Data: Automated Creation, Privacy, Bias »

It is fundamentally challenging for machine learning models to generalize to out-of-distribution data, in part due to spurious correlations. We first give a principled analysis by bounding the generalization risk on any unseen domain. Drawing inspiration from this risk upper bound, we propose a novel Disentangled representation learning method for Domain Generalization (DDG). In contrast to traditional approaches based on domain adversarial training and domain labels, DDG jointly learns semantic and variation encoders for disentanglement while employing strong regularizations from minimizing domain divergence and promoting semantic invariance. Our method is able to effectively disentangle semantic and variation factors. Such a disentanglement enables us to easily manipulate and augment the training data. Leveraging the augmented training data, DDG learns intrinsic representations of semantic concepts that are invariant to nuisance factors and generalize across different domains. Comprehensive experiments on a number of benchmarks show that DDG can achieve state-of-the-art performance on the task of domain generalization and uncover interpretable salient structure within data.

Keywords: Architectures

Author Information

Hanlin Zhang ( Carnegie Mellon University)
Yi-Fan Zhang (NLPR, China)
Weiyang Liu (University of Cambridge)
Adrian Weller (University of Cambridge, Alan Turing Institute)
Adrian Weller

Adrian Weller is Programme Director for AI at The Alan Turing Institute, the UK national institute for data science and AI, and is a Turing AI Fellow leading work on trustworthy Machine Learning (ML). He is a Principal Research Fellow in ML at the University of Cambridge, and at the Leverhulme Centre for the Future of Intelligence where he is Programme Director for Trust and Society. His interests span AI, its commercial applications and helping to ensure beneficial outcomes for society. Previously, Adrian held senior roles in finance. He received a PhD in computer science from Columbia University, and an undergraduate degree in mathematics from Trinity College, Cambridge.

Bernhard Schölkopf (Max Planck Institute for Intelligent Systems)
Eric Xing (Petuum Inc. and CMU)

More from the Same Authors