Workshop
Spurious correlations, Invariance, and Stability (SCIS)
Aahlad Puli · Maggie Makar · Victor Veitch · Yoav Wald · Mark Goldstein · Limor Gultchin · Angela Zhou · Uri Shalit · Suchi Saria
Room 340 - 342
Fri 22 Jul, 5:45 a.m. PDT
Machine learning models often break when deployed in the wild, despite excellent performance on benchmarks. In particular, models can learn to rely on apparently unnatural or irrelevant features. For instance, 1) in detecting lung disease from chest X-rays, models rely on the type of scanner rather than physiological signals, 2) in natural language inference, models rely on the number of shared words rather than the subject’s relationship with the object, 3) in precision medicine, polygenic risk scores for diseases like breast cancer rely on genes prevalent mainly in European populations, and predict poorly in other populations. In examples like these and others, the undesirable behavior stems from the model exploiting a spurious correlation. Improper treatment of spurious correlations can discourage the use of ML in the real world and lead to catastrophic consequences in extreme cases. The recent surge of interest in this issue is accordingly welcome and timely: more than 50 closely related papers have been published just in ICML 2021, NeurIPS 2021, and ICLR 2022. However, the most fundamental questions remain unanswered— e.g., how should the notion of spurious correlations be made precise? How should one evaluate models in the presence of spurious correlations? In which situations can a given method be expected to work, or fail? Which notions of invariance are fruitful and tractable? Further, relevant work has sprung up ad hoc from several distinct communities, with limited interplay between them: invariance and independence-constrained learning in causality-inspired ML, methods to decorrelate predictions and protected features (e.g. race) in algorithmic fairness, and stress testing procedures to discover unexpected model dependencies in reliable ML. This workshop will bring together these different communities to make progress on common foundational problems, and facilitate their interaction with domain-experts to build impactful collaborations.
Schedule
Fri 5:45 a.m. - 6:00 a.m.
|
Introductory Remarks
(
Presentation
)
>
SlidesLive Video |
🔗 |
Fri 6:00 a.m. - 6:25 a.m.
|
Invited Talks 1, Bernhard Schölkopf and David Lopez-Paz
(
Invited Talk
)
>
SlidesLive Video |
Bernhard Schölkopf · David Lopez-Paz 🔗 |
Fri 6:55 a.m. - 7:10 a.m.
|
Invited talks I, Q/A
(
Q/A session
)
>
|
Bernhard Schölkopf · David Lopez-Paz 🔗 |
Fri 7:10 a.m. - 7:30 a.m.
|
Break
|
🔗 |
Fri 7:30 a.m. - 8:25 a.m.
|
Invited talks 2, Christina Heinze-Deml and Marzyeh Ghassemi
(
Invited talk
)
>
SlidesLive Video |
Christina Heinze-Deml · Marzyeh Ghassemi 🔗 |
Fri 8:25 a.m. - 8:40 a.m.
|
Invited talks 2 Q/A, Christina and Marzyeh
(
Q/A
)
>
|
Christina Heinze-Deml · Marzyeh Ghassemi 🔗 |
Fri 8:40 a.m. - 9:30 a.m.
|
Spotlights
(
Spotlights
)
>
SlidesLive Video |
Pratyush Maini · JIVAT NEET KAUR · Anil Palepu · Polina Kirichenko · Revant Teotia 🔗 |
Fri 9:30 a.m. - 10:30 a.m.
|
Lunch break
|
🔗 |
Fri 10:30 a.m. - 11:50 a.m.
|
Poster Session (in-person only)
(
In-person poster session
)
>
|
🔗 |
Fri 11:50 a.m. - 1:10 p.m.
|
Invited talks 3, Amy Zhang, Rich Zemel and Liting Sun
(
Invited talk
)
>
SlidesLive Video |
Amy Zhang · Richard Zemel · Liting Sun 🔗 |
Fri 1:10 p.m. - 1:30 p.m.
|
Invited talks 3, Q/A, Amy, Rich and Liting
(
Live Q/A session
)
>
|
Liting Sun · Amy Zhang · Richard Zemel 🔗 |
Fri 1:35 p.m. - 2:35 p.m.
|
SCIS 2022 Panel
(
Live panel over zoom
)
>
SlidesLive Video |
🔗 |
Fri 2:40 p.m. - 2:45 p.m.
|
Closing remarks
(
Presentation
)
>
SlidesLive Video |
🔗 |
Fri 2:45 p.m. - 4:30 p.m.
|
Poster Session (in-person only)
(
In-person poster session
)
>
|
🔗 |
Fri 2:45 p.m. - 3:30 p.m.
|
Breakout sessions
(
Breakout sessions (in-person and virtual)
)
>
|
🔗 |
-
|
Towards Better Understanding of Self-Supervised Representations ( Poster ) > link | Neha Mukund Kalibhat · Kanika Narang · Hamed Firooz · Maziar Sanjabi · Soheil Feizi 🔗 |
-
|
Causal Balancing for Domain Generalization ( Poster ) > link | Xinyi Wang · Michael Saxon · Jiachen Li · Hongyang Zhang · Kun Zhang · William Wang 🔗 |
-
|
In the Eye of the Beholder: Robust Prediction with Causal User Modeling ( Poster ) > link | Amir Feder · Guy Horowitz · Yoav Wald · Roi Reichart · Nir Rosenfeld 🔗 |
-
|
Causal Prediction Can Induce Performative Stability ( Poster ) > link | Bogdan Kulynych 🔗 |
-
|
Evaluating and Improving Robustness of Self-Supervised Representations to Spurious Correlations
(
Poster
)
>
link
SlidesLive Video |
Kimia Hamidieh · Haoran Zhang · Marzyeh Ghassemi 🔗 |
-
|
Domain Adaptation under Open Set Label Shift
(
Poster
)
>
link
SlidesLive Video |
Saurabh Garg · Sivaraman Balakrishnan · Zachary Lipton 🔗 |
-
|
Towards Domain Adversarial Methods to Mitigate Texture Bias
(
Poster
)
>
link
SlidesLive Video |
Dhruva Kashyap · Sumukh K Aithal · Rakshith C · Natarajan Subramanyam 🔗 |
-
|
Modeling the Data-Generating Process is Necessary for Out-of-Distribution Generalization ( Poster ) > link | JIVAT NEET KAUR · Emre Kiciman · Amit Sharma 🔗 |
-
|
Invariance Discovery for Systematic Generalization in Reinforcement Learning ( Poster ) > link | Mirco Mutti · Riccardo De Santi · Emanuele Rossi · Juan Calderon · Michael Bronstein · Marcello Restelli 🔗 |
-
|
Probing Classifiers are Unreliable for Concept Removal and Detection ( Poster ) > link | Abhinav Kumar · Chenhao Tan · Amit Sharma 🔗 |
-
|
Are We Viewing the Problem of Robust Generalisation through the Appropriate Lens? ( Poster ) > link | Mohamed Omran · Bernt Schiele 🔗 |
-
|
Detecting Shortcut Learning using Mutual Information
(
Poster
)
>
link
SlidesLive Video |
Mohammed Adnan · Yani Ioannou · Chuan-Yung Tsai · Angus Galloway · Hamid Tizhoosh · Graham Taylor 🔗 |
-
|
Selection Bias Induced Spurious Correlations in Large Language Models
(
Poster
)
>
link
SlidesLive Video |
Emily McMilin 🔗 |
-
|
Invariance Principle Meets Out-of-Distribution Generalization on Graphs
(
Poster
)
>
link
SlidesLive Video |
Yongqiang Chen · Yonggang Zhang · Yatao Bian · Han Yang · Kaili MA · Binghui Xie · Tongliang Liu · Bo Han · James Cheng 🔗 |
-
|
Latent Variable Models for Bayesian Causal Discovery
(
Poster
)
>
link
SlidesLive Video |
Jithendaraa Subramanian · Jithendaraa Subramanian · Yashas Annadani · Ivaxi Sheth · Stefan Bauer · Derek Nowrouzezahrai · Samira Ebrahimi Kahou 🔗 |
-
|
Understanding Generalization and Robustess of Learned Representations of Chaotic Dynamical Systems ( Poster ) > link | Luã Streit · Vikram Voleti · Tegan Maharaj 🔗 |
-
|
Policy Architectures for Compositional Generalization in Control ( Poster ) > link | Allan Zhou · Vikash Kumar · Chelsea Finn · Aravind Rajeswaran 🔗 |
-
|
Representation Learning as Finding Necessary and Sufficient Causes ( Poster ) > link | Yixin Wang · Michael Jordan 🔗 |
-
|
Unsupervised Learning under Latent Label Shift
(
Poster
)
>
link
SlidesLive Video |
Pranav Mani · Manley Roberts · Saurabh Garg · Zachary Lipton 🔗 |
-
|
A Study of Causal Confusion in Preference-Based Reward Learning
(
Poster
)
>
link
SlidesLive Video |
Jeremy Tien · Zhiyang He · Zackory Erickson · Anca Dragan · Daniel S Brown 🔗 |
-
|
Learning to induce causal structure ( Poster ) > link | Rosemary Nan Ke · Silvia Chiappa · Jane Wang · Jorg Bornschein · Anirudh Goyal · Melanie Rey · Matthew Botvinick · Theophane Weber · Michael Mozer · Danilo J. Rezende 🔗 |
-
|
Repeated Environment Inference for Invariant Learning ( Poster ) > link | Aayush Mishra · Anqi Liu 🔗 |
-
|
Finding Spuriously Correlated Visual Attributes
(
Poster
)
>
link
SlidesLive Video |
Revant Teotia · Chengzhi Mao · Carl Vondrick 🔗 |
-
|
BARACK: Partially Supervised Group Robustness With Guarantees
(
Poster
)
>
link
SlidesLive Video |
Nimit Sohoni · Maziar Sanjabi · Nicolas Ballas · Aditya Grover · Shaoliang Nie · Hamed Firooz · Christopher Re 🔗 |
-
|
Towards Environment-Invariant Representation Learning for Robust Task Transfer
(
Poster
)
>
link
SlidesLive Video |
Benjamin Eyre · Richard Zemel · Elliot Creager 🔗 |
-
|
Doubly Right Object Recognition
(
Poster
)
>
link
SlidesLive Video |
Revant Teotia · Chengzhi Mao · Carl Vondrick 🔗 |
-
|
SimpleSpot and Evaluating Systemic Errors using Synthetic Image Datasets ( Poster ) > link | Gregory Plumb · Nari Johnson · Ángel Alexander Cabrera · Marco Ribeiro · Ameet Talwalkar 🔗 |
-
|
"Why did the Model Fail?": Attributing Model Performance Changes to Distribution Shifts ( Poster ) > link | Haoran Zhang · Harvineet Singh · Shalmali Joshi 🔗 |
-
|
Characterizing Datapoints via Second-Split Forgetting ( Poster ) > link | Pratyush Maini · Saurabh Garg · Zachary Lipton · Zico Kolter 🔗 |
-
|
Invariant and Transportable Representations for Anti-Causal Domain Shifts ( Poster ) > link | Yibo Jiang · Victor Veitch 🔗 |
-
|
Contrastive Adapters for Foundation Model Group Robustness ( Poster ) > link | Michael Zhang · Christopher Re 🔗 |
-
|
HyperInvariances: Amortizing Invariance Learning
(
Poster
)
>
link
SlidesLive Video |
Ruchika Chavhan · Henry Gouk · Jan Stuehmer · Timothy Hospedales 🔗 |
-
|
Conditional Distributional Invariance through Implicit Regularization ( Poster ) > link | Tanmay Gupta 🔗 |
-
|
Enhancing Unit-tests for Invariance Discovery
(
Poster
)
>
link
SlidesLive Video |
Piersilvio De Bartolomeis · Antonio Orvieto · Giambattista Parascandolo 🔗 |
-
|
Diversify and Disambiguate: Learning from Underspecified Data
(
Poster
)
>
link
SlidesLive Video |
Yoonho Lee · Huaxiu Yao · Chelsea Finn 🔗 |
-
|
Unsupervised Causal Generative Understanding of Images
(
Poster
)
>
link
SlidesLive Video |
Titas Anciukevičius · Patrick Fox-Roberts · Edward Rosten · Paul Henderson 🔗 |
-
|
Causal Discovery using Model Invariance through Knockoff Interventions
(
Poster
)
>
link
SlidesLive Video |
Wasim Ahmad · Maha Shadaydeh · Joachim Denzler 🔗 |
-
|
Using causal modeling to analyze generalization of biomarkers in high-dimensional domains: a case study of adaptive immune repertoires
(
Poster
)
>
link
SlidesLive Video |
Milena Pavlović · Ghadi S. Al Hajj · Victor Greiff · Johan Pensar · Geir Kjetil Sandve 🔗 |
-
|
The Importance of Background Information for Out of Distribution Generalization
(
Poster
)
>
link
SlidesLive Video |
Jupinder Parmar · Khaled Saab · Brian Pogatchnik · Daniel Rubin · Christopher Ré 🔗 |
-
|
Self-Supervision on Images and Text Reduces Reliance on Visual Shortcut Features ( Poster ) > link | Anil Palepu · Andrew Beam 🔗 |
-
|
Out-of-Distribution Failure through the Lens of Labeling Mechanisms: An Information Theoretic Approach
(
Poster
)
>
link
SlidesLive Video |
Soroosh Shahtalebi · Zining Zhu · Frank Rudzicz 🔗 |
-
|
How much Data is Augmentation Worth?
(
Poster
)
>
link
SlidesLive Video |
Jonas Geiping · Gowthami Somepalli · Ravid Shwartz-Ziv · Andrew Wilson · Tom Goldstein · Micah Goldblum 🔗 |
-
|
On the Generalization and Adaption Performance of Causal Models ( Poster ) > link | Nino Scherrer · Anirudh Goyal · Stefan Bauer · Yoshua Bengio · Rosemary Nan Ke 🔗 |
-
|
Learning Switchable Representation with Masked Decoding and Sparse Encoding
(
Poster
)
>
link
SlidesLive Video |
Kohei Hayashi · Masanori Koyama 🔗 |
-
|
Improving Group-based Robustness and Calibration via Ordered Risk and Confidence Regularization ( Poster ) > link | Seungjae Shin · Byeonghu Na · HeeSun Bae · JoonHo Jang · Hyemi Kim · Kyungwoo Song · Youngjae Cho · IL CHUL MOON 🔗 |
-
|
Towards Group Robustness in the Presence of Partial Group Labels
(
Poster
)
>
link
SlidesLive Video |
Vishnu Lokhande · Kihyuk Sohn · Jinsung Yoon · Madeleine Udell · Chen-Yu Lee · Tomas Pfister 🔗 |
-
|
Towards Multi-level Fairness and Robustness on Federated Learning ( Poster ) > link | Fengda Zhang · Kun Kuang · Yuxuan Liu · Long Chen · Jiaxun Lu · Yunfeng Shao · Fei Wu · Chao Wu · Jun Xiao 🔗 |
-
|
Learning Debiased Classifier with Biased Committee ( Poster ) > link | Nayeong Kim · SEHYUN HWANG · Sungsoo Ahn · Jaesik Park · Suha Kwak 🔗 |
-
|
Causal Omnivore: Fusing Noisy Estimates of Spurious Correlations
(
Poster
)
>
link
SlidesLive Video |
Dyah Adila · Sonia Cromp · SICHENG MO · Frederic Sala 🔗 |
-
|
Robust Calibration with Multi-domain Temperature Scaling ( Poster ) > link | Yaodong Yu · Stephen Bates · Yi Ma · Michael Jordan 🔗 |
-
|
A Unified Causal View of Domain Invariant Representation Learning ( Poster ) > link | Zihao Wang · Victor Veitch 🔗 |
-
|
Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations ( Poster ) > link | Polina Kirichenko · Polina Kirichenko · Pavel Izmailov · Andrew Wilson 🔗 |
-
|
Evaluating Robustness to Dataset Shift via Parametric Robustness Sets ( Poster ) > link | Michael Oberst · Nikolaj Thams · David Sontag 🔗 |
-
|
Causally motivated multi-shortcut identification and removal ( Poster ) > link | Jiayun Zheng · Maggie Makar 🔗 |
-
|
How robust are pre-trained models to distribution shift? ( Poster ) > link | Yuge Shi · Imant Daunhawer · Julia Vogt · Phil Torr · Amartya Sanyal 🔗 |
-
|
Understanding Rare Spurious Correlations in Neural Networks ( Poster ) > link | Yao-Yuan Yang · Chi-Ning Chou · Kamalika Chaudhuri 🔗 |
-
|
Optimization-based Causal Estimation from Heterogenous Environments ( Poster ) > link | Mingzhang Yin · Yixin Wang · David Blei 🔗 |
-
|
Automated Invariance Testing for Machine Learning Models Using Sparse Linear Layers ( Poster ) > link | Zukang Liao · Michael Cheung 🔗 |
-
|
Fairness and robustness in anti-causal prediction ( Poster ) > link | Maggie Makar · Alexander D'Amour 🔗 |
-
|
Are Vision Transformers Robust to Spurious Correlations ?
(
Poster
)
>
link
SlidesLive Video |
Soumya Suvra Ghosal · Yifei Ming · Sharon Li 🔗 |
-
|
DAFT: Distilling Adversarially Fine-tuned teachers for OOD Robustness
(
Poster
)
>
link
SlidesLive Video |
Anshul Nasery · Sravanti Addepalli · Praneeth Netrapalli · Prateek Jain 🔗 |
-
|
On the nonlinear correlation of ML performance across data subpopulations
(
Poster
)
>
link
SlidesLive Video |
Weixin Liang · Yining Mao · Yongchan Kwon · Xinyu Yang · James Zou 🔗 |
-
|
Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty ( Poster ) > link | Thomas George · Guillaume Lajoie · Aristide Baratin 🔗 |
-
|
OOD-Probe: A Neural Interpretation of Out-of-Domain Generalization
(
Poster
)
>
link
SlidesLive Video |
Zining Zhu · Soroosh Shahtalebi · Frank Rudzicz 🔗 |
-
|
Linear Connectivity Reveals Generalization Strategies
(
Poster
)
>
link
SlidesLive Video |
Jeevesh Juneja · Rachit Bansal · Kyunghyun Cho · João Sedoc · Naomi Saphra 🔗 |
-
|
SelecMix: Debiased Learning by Mixing up Contradicting Pairs ( Poster ) > link | Inwoo Hwang · Sangjun Lee · Yunhyeok Kwak · Seong Joon Oh · Damien Teney · Jin-Hwa Kim · Byoung-Tak Zhang 🔗 |
-
|
Optimizing maintenance by learning individual treatment effects ( Poster ) > link | Toon Vanderschueren · Robert Boute · Tim Verdonck · Bart Baesens · Wouter Verbeke 🔗 |