Timezone: »
Evaluating Robustness to Dataset Shift via Parametric Robustness Sets
Nikolaj Thams · Michael Oberst · David Sontag
We give a method for proactively identifying small, plausible shifts in distribution which lead to large differences in model performance. To ensure that these shifts are plausible, we parameterize them in terms of interpretable changes in causal mechanisms of observed variables. This defines a parametric robustness set of plausible distributions and a corresponding worst-case loss. We construct a local approximation to the loss under shift, and show that problem of finding worst-case shifts can be efficiently solved.
Author Information
Nikolaj Thams (University of Copenhagen)
Michael Oberst (MIT)
David Sontag (Massachusetts Institute of Technology)
More from the Same Authors
-
2021 : Invariant Policy Learning: A Causal Perspective »
Sorawit Saengkyongam · Nikolaj Thams · Jonas Peters · Niklas Pfister -
2022 : Evaluating Robustness to Dataset Shift via Parametric Robustness Sets »
Michael Oberst · Nikolaj Thams · David Sontag -
2022 Poster: Invariant Ancestry Search »
Phillip Bredahl Mogensen · Nikolaj Thams · Jonas Peters -
2022 Spotlight: Invariant Ancestry Search »
Phillip Bredahl Mogensen · Nikolaj Thams · Jonas Peters -
2022 Poster: Sample Efficient Learning of Predictors that Complement Humans »
Mohammad-Amin Charusaie · Hussein Mozannar · David Sontag · Samira Samadi -
2022 Poster: Co-training Improves Prompt-based Learning for Large Language Models »
Hunter Lang · Monica Agrawal · Yoon Kim · David Sontag -
2022 Spotlight: Sample Efficient Learning of Predictors that Complement Humans »
Mohammad-Amin Charusaie · Hussein Mozannar · David Sontag · Samira Samadi -
2022 Spotlight: Co-training Improves Prompt-based Learning for Large Language Models »
Hunter Lang · Monica Agrawal · Yoon Kim · David Sontag -
2021 Poster: Neural Pharmacodynamic State Space Modeling »
Zeshan Hussain · Rahul G. Krishnan · David Sontag -
2021 Poster: Regularizing towards Causal Invariance: Linear Models with Proxies »
Michael Oberst · Nikolaj Thams · Jonas Peters · David Sontag -
2021 Poster: Graph Cuts Always Find a Global Optimum for Potts Models (With a Catch) »
Hunter Lang · David Sontag · Aravindan Vijayaraghavan -
2021 Spotlight: Regularizing towards Causal Invariance: Linear Models with Proxies »
Michael Oberst · Nikolaj Thams · Jonas Peters · David Sontag -
2021 Oral: Graph Cuts Always Find a Global Optimum for Potts Models (With a Catch) »
Hunter Lang · David Sontag · Aravindan Vijayaraghavan -
2021 Spotlight: Neural Pharmacodynamic State Space Modeling »
Zeshan Hussain · Rahul G. Krishnan · David Sontag -
2020 Poster: Estimation of Bounds on Potential Outcomes For Decision Making »
Maggie Makar · Fredrik Johansson · John Guttag · David Sontag -
2020 Poster: Empirical Study of the Benefits of Overparameterization in Learning Latent Variable Models »
Rares-Darius Buhai · Yoni Halpern · Yoon Kim · Andrej Risteski · David Sontag -
2020 Poster: Consistent Estimators for Learning to Defer to an Expert »
Hussein Mozannar · David Sontag -
2019 Poster: Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models »
Michael Oberst · David Sontag -
2019 Oral: Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models »
Michael Oberst · David Sontag -
2018 Poster: Semi-Amortized Variational Autoencoders »
Yoon Kim · Sam Wiseman · Andrew Miller · David Sontag · Alexander Rush -
2018 Oral: Semi-Amortized Variational Autoencoders »
Yoon Kim · Sam Wiseman · Andrew Miller · David Sontag · Alexander Rush -
2017 Poster: Estimating individual treatment effect: generalization bounds and algorithms »
Uri Shalit · Fredrik D Johansson · David Sontag -
2017 Talk: Estimating individual treatment effect: generalization bounds and algorithms »
Uri Shalit · Fredrik D Johansson · David Sontag -
2017 Poster: Simultaneous Learning of Trees and Representations for Extreme Classification and Density Estimation »
Yacine Jernite · Anna Choromanska · David Sontag -
2017 Talk: Simultaneous Learning of Trees and Representations for Extreme Classification and Density Estimation »
Yacine Jernite · Anna Choromanska · David Sontag