Timezone: »
Poster
Representer Point Selection for Explaining Regularized High-dimensional Models
Che-Ping Tsai · Jiong Zhang · Hsiang-Fu Yu · Eli Chien · Cho-Jui Hsieh · Pradeep Ravikumar
We introduce a novel class of sample-based explanations we term *high-dimensional representers*, that can be used to explain the predictions of a regularized high-dimensional model in terms of importance weights for each of the training samples. Our workhorse is a novel representer theorem for general regularized high-dimensional models, which decomposes the model prediction in terms of contributions from each of the training samples: with positive (negative) values corresponding to positive (negative) impact training samples to the model's prediction. We derive consequences for the canonical instances of $\ell_1$ regularized sparse models and nuclear norm regularized low-rank models. As a case study, we further investigate the application of low-rank models in the context of collaborative filtering, where we instantiate high-dimensional representers for specific popular classes of models. Finally, we study the empirical performance of our proposed methods on three real-world binary classification datasets and two recommender system datasets. We also showcase the utility of high-dimensional representers in explaining model recommendations.
Author Information
Che-Ping Tsai (Carnegie Mellon University)
Jiong Zhang (Amazon)
Hsiang-Fu Yu (Amazon)
Eli Chien (University of Illinois Urbana-Champaign)
Cho-Jui Hsieh (UCLA)
Pradeep Ravikumar (Carnegie Mellon University)
More from the Same Authors
-
2021 : Fast Certified Robust Training with Short Warmup »
Zhouxing Shi · Yihan Wang · Huan Zhang · Jinfeng Yi · Cho-Jui Hsieh -
2021 : Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Neural Network Robustness Verification »
Shiqi Wang · Huan Zhang · Kaidi Xu · Xue Lin · Suman Jana · Cho-Jui Hsieh · Zico Kolter -
2021 : When Is Generalizable Reinforcement Learning Tractable? »
Dhruv Malik · Yuanzhi Li · Pradeep Ravikumar -
2023 : Identifying Causal Mechanism Shifts among Nonlinear Additive Noise Models »
Tianyu Chen · Kevin Bello · Bryon Aragam · Pradeep Ravikumar -
2023 : Learning Linear Causal Representations from Interventions under General Nonlinear Mixing »
Simon Buchholz · Goutham Rajendran · Elan Rosenfeld · Bryon Aragam · Bernhard Schölkopf · Pradeep Ravikumar -
2023 : Learning Linear Causal Representations from Interventions under General Nonlinear Mixing »
Simon Buchholz · Goutham Rajendran · Elan Rosenfeld · Bryon Aragam · Bernhard Schölkopf · Pradeep Ravikumar -
2023 : Learning with Explanation Constraints »
Rattana Pukdee · Dylan Sam · Nina Balcan · Pradeep Ravikumar -
2023 : Learning Linear Causal Representations from Interventions under General Nonlinear Mixing »
Simon Buchholz · Goutham Rajendran · Elan Rosenfeld · Bryon Aragam · Bernhard Schölkopf · Pradeep Ravikumar -
2023 : Global Optimality in Bivariate Gradient-based DAG Learning »
Chang Deng · Kevin Bello · Pradeep Ravikumar · Bryon Aragam -
2023 : Formal Verification for Neural Networks with General Nonlinearities via Branch-and-Bound »
Zhouxing Shi · Qirui Jin · Huan Zhang · Zico Kolter · Suman Jana · Cho-Jui Hsieh -
2023 Workshop: 2nd Workshop on Formal Verification of Machine Learning »
Mark Müller · Brendon G. Anderson · Leslie Rice · Zhouxing Shi · Shubham Ugare · Huan Zhang · Martin Vechev · Zico Kolter · Somayeh Sojoudi · Cho-Jui Hsieh -
2023 Poster: Optimizing NOTEARS Objectives via Topological Swaps »
Chang Deng · Kevin Bello · Bryon Aragam · Pradeep Ravikumar -
2023 Poster: PINA: Leveraging Side Information in eXtreme Multi-label Classification via Predicted Instance Neighborhood Aggregation »
Eli Chien · Jiong Zhang · Cho-Jui Hsieh · Jyun-Yu Jiang · Wei-Cheng Chang · Olgica Milenkovic · Hsiang-Fu Yu -
2023 Poster: Scaling Up Dataset Distillation to ImageNet-1K with Constant Memory »
Justin Cui · Ruochen Wang · Si Si · Cho-Jui Hsieh -
2023 Poster: Faith-Shap: The Faithful Shapley Interaction Index »
Che-Ping Tsai · Chih-Kuan Yeh · Pradeep Ravikumar -
2022 Workshop: Workshop on Formal Verification of Machine Learning »
Huan Zhang · Leslie Rice · Kaidi Xu · aditi raghunathan · Wan-Yi Lin · Cho-Jui Hsieh · Clark Barrett · Martin Vechev · Zico Kolter -
2022 Poster: Building Robust Ensembles via Margin Boosting »
Dinghuai Zhang · Hongyang Zhang · Aaron Courville · Yoshua Bengio · Pradeep Ravikumar · Arun Sai Suggala -
2022 Poster: A Branch and Bound Framework for Stronger Adversarial Attacks of ReLU Networks »
Huan Zhang · Shiqi Wang · Kaidi Xu · Yihan Wang · Suman Jana · Cho-Jui Hsieh · Zico Kolter -
2022 Spotlight: Building Robust Ensembles via Margin Boosting »
Dinghuai Zhang · Hongyang Zhang · Aaron Courville · Yoshua Bengio · Pradeep Ravikumar · Arun Sai Suggala -
2022 Spotlight: A Branch and Bound Framework for Stronger Adversarial Attacks of ReLU Networks »
Huan Zhang · Shiqi Wang · Kaidi Xu · Yihan Wang · Suman Jana · Cho-Jui Hsieh · Zico Kolter -
2021 Poster: Overcoming Catastrophic Forgetting by Bayesian Generative Regularization »
PEI-HUNG Chen · Wei Wei · Cho-Jui Hsieh · Bo Dai -
2021 Spotlight: Overcoming Catastrophic Forgetting by Bayesian Generative Regularization »
PEI-HUNG Chen · Wei Wei · Cho-Jui Hsieh · Bo Dai -
2021 Poster: DORO: Distributional and Outlier Robust Optimization »
Runtian Zhai · Chen Dan · Zico Kolter · Pradeep Ravikumar -
2021 Spotlight: DORO: Distributional and Outlier Robust Optimization »
Runtian Zhai · Chen Dan · Zico Kolter · Pradeep Ravikumar -
2021 Poster: On Proximal Policy Optimization's Heavy-tailed Gradients »
Saurabh Garg · Joshua Zhanson · Emilio Parisotto · Adarsh Prasad · Zico Kolter · Zachary Lipton · Sivaraman Balakrishnan · Ruslan Salakhutdinov · Pradeep Ravikumar -
2021 Spotlight: On Proximal Policy Optimization's Heavy-tailed Gradients »
Saurabh Garg · Joshua Zhanson · Emilio Parisotto · Adarsh Prasad · Zico Kolter · Zachary Lipton · Sivaraman Balakrishnan · Ruslan Salakhutdinov · Pradeep Ravikumar -
2020 Poster: Uniform Convergence of Rank-weighted Learning »
Justin Khim · Liu Leqi · Adarsh Prasad · Pradeep Ravikumar -
2020 Poster: Sharp Statistical Guaratees for Adversarially Robust Gaussian Classification »
Chen Dan · Yuting Wei · Pradeep Ravikumar -
2020 Poster: On Lp-norm Robustness of Ensemble Decision Stumps and Trees »
Yihan Wang · Huan Zhang · Hongge Chen · Duane Boning · Cho-Jui Hsieh -
2020 Poster: Learning to Encode Position for Transformer with Continuous Dynamical Model »
Xuanqing Liu · Hsiang-Fu Yu · Inderjit Dhillon · Cho-Jui Hsieh -
2020 Poster: Class-Weighted Classification: Trade-offs and Robust Approaches »
Ziyu Xu · Chen Dan · Justin Khim · Pradeep Ravikumar -
2020 Poster: Extreme Multi-label Classification from Aggregated Labels »
Yanyao Shen · Hsiang-Fu Yu · Sujay Sanghavi · Inderjit Dhillon -
2020 Poster: Certified Robustness to Label-Flipping Attacks via Randomized Smoothing »
Elan Rosenfeld · Ezra Winston · Pradeep Ravikumar · Zico Kolter -
2020 Poster: Stabilizing Differentiable Architecture Search via Perturbation-based Regularization »
Xiangning Chen · Cho-Jui Hsieh -
2019 Poster: Robust Decision Trees Against Adversarial Examples »
Hongge Chen · Huan Zhang · Duane Boning · Cho-Jui Hsieh -
2019 Oral: Robust Decision Trees Against Adversarial Examples »
Hongge Chen · Huan Zhang · Duane Boning · Cho-Jui Hsieh -
2018 Poster: Learning long term dependencies via Fourier recurrent units »
Jiong Zhang · Yibo Lin · Zhao Song · Inderjit Dhillon -
2018 Poster: Binary Classification with Karmic, Threshold-Quasi-Concave Metrics »
Bowei Yan · Sanmi Koyejo · Kai Zhong · Pradeep Ravikumar -
2018 Poster: Loss Decomposition for Fast Learning in Large Output Spaces »
En-Hsu Yen · Satyen Kale · Felix Xinnan Yu · Daniel Holtmann-Rice · Sanjiv Kumar · Pradeep Ravikumar -
2018 Oral: Binary Classification with Karmic, Threshold-Quasi-Concave Metrics »
Bowei Yan · Sanmi Koyejo · Kai Zhong · Pradeep Ravikumar -
2018 Oral: Loss Decomposition for Fast Learning in Large Output Spaces »
En-Hsu Yen · Satyen Kale · Felix Xinnan Yu · Daniel Holtmann-Rice · Sanjiv Kumar · Pradeep Ravikumar -
2018 Oral: Learning long term dependencies via Fourier recurrent units »
Jiong Zhang · Yibo Lin · Zhao Song · Inderjit Dhillon -
2018 Poster: Stabilizing Gradients for Deep Neural Networks via Efficient SVD Parameterization »
Jiong Zhang · Qi Lei · Inderjit Dhillon -
2018 Poster: Deep Density Destructors »
David Inouye · Pradeep Ravikumar -
2018 Oral: Stabilizing Gradients for Deep Neural Networks via Efficient SVD Parameterization »
Jiong Zhang · Qi Lei · Inderjit Dhillon -
2018 Oral: Deep Density Destructors »
David Inouye · Pradeep Ravikumar -
2017 Poster: Ordinal Graphical Models: A Tale of Two Approaches »
ARUN SAI SUGGALA · Eunho Yang · Pradeep Ravikumar -
2017 Poster: Doubly Greedy Primal-Dual Coordinate Descent for Sparse Empirical Risk Minimization »
Qi Lei · En-Hsu Yen · Chao-Yuan Wu · Inderjit Dhillon · Pradeep Ravikumar -
2017 Poster: Latent Feature Lasso »
En-Hsu Yen · Wei-Cheng Lee · Sung-En Chang · Arun Suggala · Shou-De Lin · Pradeep Ravikumar -
2017 Talk: Doubly Greedy Primal-Dual Coordinate Descent for Sparse Empirical Risk Minimization »
Qi Lei · En-Hsu Yen · Chao-Yuan Wu · Inderjit Dhillon · Pradeep Ravikumar -
2017 Talk: Ordinal Graphical Models: A Tale of Two Approaches »
ARUN SAI SUGGALA · Eunho Yang · Pradeep Ravikumar -
2017 Talk: Latent Feature Lasso »
En-Hsu Yen · Wei-Cheng Lee · Sung-En Chang · Arun Suggala · Shou-De Lin · Pradeep Ravikumar