Spurious Correlations and Where to Find Them

Despite being a well-known drawback of data-driven learning and having several algorithms proposed to mitigate it, we are yet to jointly derive the indicators of spurious correlations. As a result, the solutions built upon standalone hypotheses fail to beat simple ERM baselines. We collect some of the commonly studied hypotheses behind the occurrence of spurious correlations and study their influence on preliminary ERM baselines using synthetic datasets generated using causal graphs. Subsequently, we observe patterns connecting these hypotheses and model design choices.