Towards Environment-Invariant Representation Learning for Robust Task Transfer

To train a classification model that is robust to distribution shifts upon deployment, auxiliary labels indicating the various ``environments'' of data collection can be leveraged to mitigate reliance on environment-specific features. This paper investigates how to evaluate whether a model has formed environment-invariant representations, and proposes an objective that encourages learning such representations, as opposed to an invariant classifier. We also introduce a novel paradigm for evaluating environment-invariant performance, to determine if learned representations can robustly transfer to a new task.