Keywords: • Model uncertainty estimation and calibration • Probabilistic (Bayesian and non-Bayesian) neural networks • Anomaly detection and out-of-distribution detection • Robustness to distribution shift • Model misspecification • Quantifying different types of uncertainty • Open world recognition
There has been growing interest in rectifying deep neural network instabilities. Challenges arise when models receive samples drawn from outside the training distribution. For example, a neural network tasked with classifying handwritten digits may assign high confidence predictions to cat images. Anomalies are frequently encountered when deploying ML models in the real world. Well-calibrated predictive uncertainty estimates are indispensable for many machine learning applications, such as self-driving vehicles and medical diagnosis systems. Generalization to unseen and worst-case inputs is also essential for robustness to distributional shift. In order to have ML models reliably predict in open environment, we must deepen technical understanding in the emerging areas of: (1) learning algorithms that can detect changes in data distribution (e.g. out-of-distribution examples); (2) mechanisms to estimate and calibrate confidence produced by neural networks in typical and unforeseen scenarios; (3) methods to improve out-of-distribution generalization, including generalization to temporal, geographical, hardware, adversarial, and image-quality changes; (4) benchmark datasets and protocols for evaluating model performance under distribution shift; and (5) key applications of robust and uncertainty-aware deep learning (e.g., computer vision, robotics, self-driving vehicles, medical imaging) as well as broader machine learning tasks.
This workshop will bring together researchers and practitioners from the machine learning communities, and highlight recent work that contributes to addressing these challenges. Our agenda will feature contributed papers with invited speakers. Through the workshop we hope to help identify fundamentally important directions on robust and reliable deep learning, and foster future collaborations.
Fri 7:30 a.m. - 7:40 a.m.
|
Opening Remarks
(
Presentation
)
link »
SlidesLive Video » https://slideslive.com/38930571/opening-remarks-workshop-on-robustness-uncertainty-in-deep-learning |
Sharon Yixuan Li 🔗 |
Fri 7:40 a.m. - 8:10 a.m.
|
Keynote #1 Matthias Hein
(
Keynote
)
link »
SlidesLive Video » http://slideslive.com/38930572 |
Matthias Hein 🔗 |
Fri 8:10 a.m. - 8:15 a.m.
|
Spotlight Talk 1: Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder
(
Spotlight
)
link »
SlidesLive Video » https://slideslive.com/38930944 |
Zhisheng Xiao 🔗 |
Fri 8:15 a.m. - 8:20 a.m.
|
Spotlight Talk 2: A Closer Look at Accuracy vs. Robustness
(
Spotlight
)
link »
SlidesLive Video » https://slideslive.com/38930945 |
Yao-Yuan Yang 🔗 |
Fri 8:20 a.m. - 8:25 a.m.
|
Spotlight Talk 3: Depth Uncertainty in Neural Networks
(
Spotlight
)
link »
SlidesLive Video » https://slideslive.com/38930946 |
Javier Antorán · James Allingham 🔗 |
Fri 8:25 a.m. - 8:30 a.m.
|
Spotlight Talk 4: Few-shot Out-of-Distribution Detection
(
Spotlight
)
link »
SlidesLive Video » https://slideslive.com/38930947 |
Kuan-Chieh Wang 🔗 |
Fri 8:30 a.m. - 8:35 a.m.
|
Spotlight Talk 5: Detecting Failure Modes in Image Reconstructions with Interval Neural Network Uncertainty
(
Spotlight
)
link »
SlidesLive Video » https://slideslive.com/38930948 |
Luis Oala 🔗 |
Fri 8:35 a.m. - 8:40 a.m.
|
Spotlight Talk 6: On using Focal Loss for Neural Network Calibration
(
Spotlight
)
link »
SlidesLive Video » https://slideslive.com/38930949 |
Jishnu Mukhoti 🔗 |
Fri 8:40 a.m. - 8:45 a.m.
|
Spotlight Talk 7: AutoAttack: reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
(
Spotlight
)
link »
SlidesLive Video » https://slideslive.com/38930950 |
Francesco Croce 🔗 |
Fri 8:45 a.m. - 8:50 a.m.
|
Spotlight Talk 8: Calibrated Top-1 Uncertainty estimates for classification by score based models
(
Spotlight
)
link »
SlidesLive Video » https://slideslive.com/38930951 |
Adam Oberman 🔗 |
Fri 9:00 a.m. - 10:00 a.m.
|
Poster Session (click to see links)
(
Poster
)
1 Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks David Stutz, Matthias Hein and Bernt Schiele [ protected link dropped ] 2 Improving robustness against common corruptions by covariate shift adaptation Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel and Matthias Bethge https://meet.google.com/ojt-atoh-wup 3 A Unified View of Label Shift Estimation Saurabh Garg, Yifan Wu, Sivaraman Balakrishnan and Zachary Lipton https://cmu.zoom.us/j/94831507038?pwd=RVVyVy96YXNiaGtjMUxyZUJ5dVcvQT09 4 A Benchmark of Medical Out of Distribution Detection Tianshi Cao, David Yu-Tung Hui, Chin-Wei Huang and Joseph Paul Cohen https://meet.google.com/jxk-paui-gao 5 Neural Ensemble Search for Performant and Calibrated Predictions Sheheryar Zaidi, Arber Zela, Thomas Elsken, Chris Holmes, Frank Hutter and Yee Whye Teh https://meet.google.com/uxs-uuxr-uwo 6 Bayesian model averaging is suboptimal for generalization under model misspecification Andres Masegosa https://meet.google.com/mpf-chva-pgy 7 Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder Zhisheng Xiao, Qing Yan and Yali Amit https://uchicago.zoom.us/j/98748547880?pwd=Q3Y0dUVPUFphbGY4NmNJK2hwZndWUT09 Password: 370663 8 A Closer Look at Accuracy vs. Robustness Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Ruslan Salakhutdinov and Kamalika Chaudhuri https://ucsd.zoom.us/j/92778085557?pwd=STJVVWMza0RKSFZsOUVIS0h6bVl5QT09 9 Depth Uncertainty in Neural Networks Javier Antorán, James Urquhart Allingham and José Miguel Hernández-Lobato https://us02web.zoom.us/j/5419103161?pwd=eXdqWURLc3o4SktwQWZOU2pZNkliQT09 10 Few-shot Out-of-Distribution Detection Kuan-Chieh Wang, Paul Vicol, Eleni Triantafillou and Richard Zemel https://vectorinstitute.zoom.us/j/97199418667?pwd=NjBWUXhzZUwrOHFBMkxSZEd2MDh3Zz09 11 Detecting Failure Modes in Image Reconstructions with Interval Neural Network Uncertainty Luis Oala, Cosmas Heiß, Jan Macdonald, Maximilian März, Gitta Kutyniok and Wojciech Samek https://us02web.zoom.us/j/88251013741?pwd=TzlOUlJXeExzUVdSUDE5RXFkTytWZz09 12 On using Focal Loss for Neural Network Calibration Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip Torr and Puneet Dokania https://meet.google.com/ehz-rzxf-xeu 13 AutoAttack: reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks Francesco Croce and Matthias Hein [ protected link dropped ] 14 Calibrated Top-1 Uncertainty estimates for classification by score based models Adam Oberman, Chris Finlay, Alexander Iannantuono and Tiago Salvador https://mcgill.zoom.us/j/91308261627 15 Bayesian Deep Ensembles via the Neural Tangent Kernel Bobby He, Balaji Lakshminarayanan and Yee Whye Teh https://meet.google.com/exb-jkju-vbr 16 The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt and Justin Gilmer https://berkeley.zoom.us/j/4219480859?pwd=QkNIYmNCZjlobEUxQ2Q5TzR4Qm1QQT09 17 Measuring Robustness to Natural Distribution Shifts in Image Classification Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht and Ludwig Schmidt https://berkeley.zoom.us/j/97514838599?pwd=SXdiOSthMzNUV01QYU5lVElFTTM2dz09 18 Redundant features can hurt robustness to distribution shift Guillermo Ortiz-Jimenez, Apostolos Modas, Seyed-Mohsen Moosavi-Dezfooli and Pascal Frossard https://epfl.zoom.us/j/99757698661?pwd=NVc5OUVHenFYc2pXVlFYbk12U0F4dz09 548657 19 Scalable Training with Information Bottleneck Objectives Andreas Kirsch, Clare Lyle and Yarin Gal https://meet.google.com/mzn-pjuh-kia 20 How Does Early Stopping Help Generalization Against Label Noise? Hwanjun Song Hwanjun, Minseok Kim, Dongmin Park and Jae-Gil Lee [ protected link dropped ] 21 Soft Labeling Affects Out-of-Distribution Detection of Deep Neural Networks Doyup Lee and Yeongjae Cheon [ protected link dropped ] 0deFHw 22 Learning Robust Representations with Score Invariant Learning Daksh Idnani and Jonathan Kao https://ucla.zoom.us/j/98883773408?pwd=Y3M2WmxNTW1HOS9hMlBZYmJJSHZ0dz09 23 PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction Sangdon Park, Osbert Bastani, Nikolai Matni and Insup Lee https://us04web.zoom.us/j/76965715700?pwd=U2tNWVBMd0RKMHcrNGlTa0VMQkY3dz09 24 Learned Uncertainty-Aware (LUNA) Bases for Bayesian Regression using Multi-Headed Auxiliary Networks Sujay Thakur, Cooper Lorsung, Yaniv Yacoby, Finale Doshi-Velez and Weiwei Pan https://harvard.zoom.us/j/96201359957?pwd=N3Q3RVJJZ0R0TmRuejYxTDRDMjAxUT09 25 Domain Generalization using Causal Matching Divyat Mahajan, Shruti Tople and Amit Sharma https://us02web.zoom.us/j/87167858273?pwd=V09XWTdjMzVSTHplcnJ6dCtpUWphUT09 26 Hydra: Preserving Ensemble Diversity for Model Distillation Linh Tran, Bastiaan S. Veeling, Kevin Roth, Jakub Swiatkowski, Joshua V. Dillon, Stephan Mandt, Jasper Snoek, Tim Salimans, Sebastian Nowozin and Rodolphe Jenatton meet.google.com/vqj-zckz-hbr 27 Predicting with High Correlation Features Devansh Arpit, Caiming Xiong and Richard Socher https://us04web.zoom.us/j/3643443782 28 Consistency Regularization for Certified Robustness of Smoothed Classifiers Jongheon Jeong and Jinwoo Shin https://us02web.zoom.us/j/85431063603?pwd=MGJ4YzR4Q2FXSWM4c3c3NURiR3NqZz09 29 DIBS: Diversity inducing Information Bottleneck in Model Ensembles Samarth Sinha, Homanga Bharadhwaj, Anirudh Goyal, Hugo Larochelle, Animesh Garg and Florian Shkurti https://utoronto.zoom.us/j/6995996022?pwd=UUIxNUNnQThLVmhLV0h5RDVJRG52dz09 30 On the relationship between class selectivity, dimensionality, and robustness Matthew Leavitt and Ari Morcos https://fb.zoom.us/j/98555977785 949102 31 Robust Variational Autoencoder for Tabular Data with β Divergence Haleh Aydore, Sergul Aydore, Richard Leahy and Anand Joshi https://usc.zoom.us/j/92860013817?pwd=aG5YekNUcFBIalV4TVNJYkhyb1UzZz09 906642 32 Uncertainty in Structured Prediction Andrey Malinin and Mark Gales https://yandex.zoom.us/j/98605165946?pwd=RkJ4RTVmYmt6eU1CcWU5RGgvNEMzdz09 33 Understanding and Improving Fast Adversarial Training Maksym Andriushchenko and Nicolas Flammarion https://epfl.zoom.us/j/98065886762?pwd=dTVFMUdFZllIZFVEMllDbkdxL2QvZz09 34 Predictive Complexity Priors Eric Nalisnick, Jonathan Gordon and Jose Miguel Hernandez Lobato https://meet.google.com/jyk-qhvg-kek 35 Provable Worst Case Guarantees for the Detection of Out-of-Distribution Data Julian Bitterwolf, Alexander Meinke and Matthias Hein [ protected link dropped ] 36 Ensemble Distribution Distillation via Regression Prior Networks Andrey Malinin, Sergey Chervontsev, Ivan Provilkov and Mark Gales meet.google.com/zbn-vwtv-eda 37 Generalizing to unseen domains via distribution matching Isabela Albuquerque, Joao Monteiro, Mohammad Darvishi, Tiago Falk and Ioannis Mitliagkas [ protected link dropped ] 2U0Qsc 38 GAN-mixup: Augmenting Across Data Manifolds for Improved Robustness Jy-Yong Sohn, Kangwook Lee, Jaekyun Moon and Dimitris Papailiopoulos https://us02web.zoom.us/j/84163398388?pwd=MGFXWm9VZGhDZ01mMXFCMkNJckdRdz09 240878 39 Improving out-of-distribution generalization via multi-task self-supervised pretraining Isabela Albuquerque, Nikhil Naik, Junnan Li, Nitish Shirish Keskar and Richard Socher [ protected link dropped ] 6byCUy 40 Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks Shreyas Padhy, Zachary Nado, Jie Ren, Jeremiah Liu, Jasper Snoek and Balaji Lakshminarayanan https://meet.google.com/cym-bnec-mbb 41 Tilted Empirical Risk Minimization Tian Li, Ahmad Beirami, Maziar Sanjabi and Virginia Smith https://cmu.zoom.us/j/96622035970?pwd=M21WMTllV0tuWkpMaUlHMDlBWmpFdz09 42 Towards Robust Classification with Deep Generative Forests Alvaro Henrique Chaim Correia, Robert Peharz and Cassio de Campos [ protected link dropped ] 43 Riemannian Continuous Normalizing Flows Emile Mathieu https://meet.google.com/vts-axeh-doo 44 Improving Calibration of BatchEnsemble with Data Augmentation Yeming Wen, Ghassen Jerfel, Rafael Muller, Michael Dusenberry, Jasper Snoek, Balaji Lakshminarayanan and Dustin Tran https://us04web.zoom.us/j/79191629966?pwd=eVNEZ2NRVGY4NEc0Uk1DK2t1Ykh6QT09. 45 Environment Inference for Invariant Learning Elliot Creager, Jörn-Henrik Jacobsen and Richard Zemel https://vectorinstitute.zoom.us/j/98611353513?pwd=Mk5xa2VqLzVBV0JzdjZINXNlWXJMdz09 470387 46 Nonlinear Gradient Estimation for Query Efficient Blackbox Attack Huichen Li, Linyi Li, Xiaojun Xu, Xiaolu Zhang, Shuang Yang and Bo Li https://illinois.zoom.us/j/92637811021?pwd=WldhQXRVaENIWXFNQjdrdXhmcG1UQT09 NLBA 47 ImageNet performance correlates with pose estimation robustness and generalization on out-of-domain data Alexander Mathis, Thomas Biasi, Mert Yüksegönül, Byron Rogers, Matthias Bethge and Mackenzie Mathis https://harvard.zoom.us/j/96654497889?pwd=Z2NrL05lQlozZndsQ25jV0JQdkRtZz09 48 CRUDE: Calibrating Regression Uncertainty Distributions Empirically Eric Zelikman, Christopher Healy, Sharon Zhou and Anand Avati https://stanford.zoom.us/j/92845303885?pwd=Kzd6d2tjMGdXL0VkK01JM09jYkVqUT09 519357 49 Positive-Unlabeled Learning with Arbitrarily Non-Representative Labeled Data Zayd Hammoudeh and Daniel Lowd https://uoregon.zoom.us/j/99760928750?pwd=dmVCUlI3WUlxNldDRGVqQmQrSDVFQT09 359623 50 Probabilistic Robustness Estimates for Deep Neural Networks Nicolas Couellan [ protected link dropped ] 51 Estimating Risk and Uncertainty in Deep Reinforcement Learning William Clements, Bastien Van Delft, Benoît-Marie Robaglia, Reda Bahi Slaoui and Sébastien Toth https://us02web.zoom.us/j/84543888187?pwd=cVdIS1ZjQUVzaTNkR0lRT2lTVzNxZz09 52 An Empirical Study of Invariant Risk Minimization Yo Joong Choe, Jiyeon Ham and Kyubyong Park https://cmu.zoom.us/j/97184088310?pwd=MGJ0bEYzQXg3RVhHSkxTTWY2RnBHdz09 53 Information-Bottleneck under Mean Field Initialization Vinayak Abrol and Jared Tanner [ protected link dropped ] 54 Adaptive Risk Minimization: A Meta-Learning Approach for Tackling Group Shift Marvin Zhang, Henrik Marklund, Abhishek Gupta, Sergey Levine and Chelsea Finn https://berkeley.zoom.us/j/96584472302?pwd=TXlPWHFzeHhVTzROYnB4KzJVSVJhUT09 55 Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift Zachary Nado, Shreyas Padhy, D. Sculley, Alexander D’amour, Balaji Lakshminarayanan and Jasper Snoek https://meet.google.com/zxr-qijn-xkx 56 Failures of Variational Autoencoders and their Effects on Downstream Tasks Yaniv Yacoby, Weiwei Pan and Finale Doshi-Velez https://harvard.zoom.us/j/98541353638?pwd=QUovTjVKYm4rM1NSamw1VUJ3eVI2Zz09 57 Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness Jeremiah Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax-Weiss and Balaji Lakshminarayanan [ protected link dropped ] 58 Bayesian BERT for Trustful Hate Speech Detection Kristian Miok, Blaz Skrlj, Daniela Zaharie and Marko Robnik-Sikonja [ protected link dropped ] 12341234 59 On the Role of Dataset Quality and Heterogeneity in Model Confidence Yuan Zhao, Jiasi Chen and Samet Oymak https://us04web.zoom.us/j/76373416589?pwd=L09lNlBDUXcvNHZ6MjV3TTJUQzUxQT0 60 Certified Adversarial Robustness via Randomized Smoothing: a Case Study for Laplace Noises Jiaye Teng, Guanghe Lee and Yang Yuan [ protected link dropped ] password 3QAa4p 61 QUEST for MEDISYN: Quasi-norm based Uncertainty ESTimation for MEDical Image SYNthesis Uddeshya Upadhyay, Viswanath P. Sudarshan and Suyash P. Awate https://monash.zoom.us/j/4586078207?pwd=OGlzeHh0bVovM3lkNCttRW40SFNMZz09 quest 62 On uncertainty estimation in active learning for image segmentation Bo Li and Tommy Sonne Alstrøm https://dtudk.zoom.us/j/65926847472?pwd=NGU0cXVlMy9oeGQ3dmp5WlFCcWgvUT09 63 Bayesian Autoencoders: Analysing and Fixing the Bernoulli likelihood for Out-of-Distribution Detection Bang Xiang Yong, Tim Pearce and Alexandra Brintrup meet.google.com/uxr-hzin-coj 64 A Comparison of Bayesian Deep Learning for Out of Distribution Detection and Uncertainty Estimation John Mitros, Arjun Pakrashi and Brian Mac Namee https://meet.google.com/puu-yysf-owt?hs=122&authuser=0 65 Practical Bayesian Neural Networks via Adaptive Subgradient Optimization Methods Samuel Kessler, Arnold Salas, Vincent Tan Weng Choon, Stefan Zohren and Stephen Roberts https://us02web.zoom.us/j/81195534206 66 Bayesian Few-Shot Classification with One-vs-Each Pólya-Gamma Augmented Gaussian Processes Jake Snell and Richard Zemel https://vectorinstitute.zoom.us/j/99320884951?pwd=aEdhbE8wUWNIS1ZuZkc4ZHFKVGNWUT09 67 Characteristics of Monte Carlo Dropout in Wide Neural Networks Joachim Sicking, Maram Akila, Tim Wirtz, Sebastian Houben and Asja Fischer https://us02web.zoom.us/j/88053290381?pwd=T2FaaXk1aUhWcS9iSFFhTW9WY1NTdz09 68 On Power Laws in Deep Ensembles Ekaterina Lobacheva, Nadezhda Chirkova, Maxim Kodryan and Dmitry Vetrov [ protected link dropped ] 69 Outlier Detection through Null Space Analysis of Neural Networks Matthew Cook, Alina Zare and Paul Gader https://ufl.zoom.us/j/94556341852?pwd=MXRoVUw5M3BUMmFYcjlGOTBVdDhIQT09 70 In a forward direction: Analyzing distribution shifts in machine translation test sets over time Thomas Liao, Benjamin Recht and Ludwig Schmidt https://berkeley.zoom.us/j/98684240218?pwd=elZ3RVVOQllkdDZGTU1PZTY0dGZPUT09 71 Characterizing Adversarial Transferability via Gradient Orthogonality and Smoothness Zhuolin Yang, Linyi Li, Xiaojun Xu, Kaizhao Liang, Shiliang Zuo, Qian Chen, Benjamin Rubinstein, Ce Zhang and Bo Li https://illinois.zoom.us/j/95161238471?pwd=NHRrZ1c0ZlAwMVp6U2Vwd3U2OXk1dz09 926964 72 Untapped Potential of Data Augmentation: A Domain Generalization Viewpoint Vihari Piratla and Shiv Shankar https://meet.google.com/xya-jmcm-htd 73 Classifying Perturbation Types for Adversarial Robustness Against Multiple Threat Models Pratyush Maini, Xinyun Chen, Bo Li and Dawn Song https://berkeley.zoom.us/j/96355581753?pwd=djFrc0hMVmQ1RU5rMzNsbCtjbXVQdz09 74 A Critical Evaluation of Open-World Machine Learning Liwei Song, Vikash Sehwag, Arjun Nitin Bhagoji and Prateek Mittal https://princeton.zoom.us/j/6441846264?pwd=RWgzYjA2aWJVbnJvaVcvc2FQRm1HUT09 75 You won't believe you can learn CIFAR-10 with this: Another take on Information Bottleneck Objectives Andreas Kirsch, Clare Lyle and Yarin Gal https://meet.google.com/tes-egvb-nso 76 Principled Uncertainty Estimation for High Dimensional Data Pascal Notin, José Miguel Hernández-Lobato and Yarin Gal https://meet.google.com/xte-ywjb-paj 77 Rethink Autoencoders: Robust Manifold Learning Taihui Li, Rishabh Mehta, Zecheng Qian and Ju Sun https://umn.zoom.us/j/92216511031?pwd=UjN1K2IvRFVvaDlpVUlXZWFqMURZUT09 7zcn93 78 Continuous-Depth Bayesian Neural Networks Winnie Xu, Ricky T.Q. Chen and David Duvenaud https://utoronto.zoom.us/j/7553398989 755996 79 Robust Temporal Point Event Localization through Smoothing and Counting Julien Schroeter, Kirill Sidorov and David Marshall https://cardiff.zoom.us/j/96449530903?pwd=RjBtNDFmY0ZlT1EyMFJRZjdNdDh0UT09 80 Chi-square Information for Invariant Learning Prasanna Sattigeri, Soumya Ghosh and Samuel Hoffman https://us04web.zoom.us/j/77772482601?pwd=MXdDTDRUWmFJbktYcWVkT3NRRWUxdz09 81 Robust Out-of-distribution Detection via Informative Outlier Mining Jiefeng Chen, Sharon Li, Xi Wu, Yingyu Liang and Somesh Jha https://stanford.zoom.us/j/99582468298?pwd=V1VINTI1U1JJalRBaktTQnk4c0VvQT09 82 An Empirical Analysis of the Impact of Data Augmentation on Distillation Deepan Das, Haley Massa, Abhimanyu Kulkarni and Theodoros Rekatsinas https://meet.google.com/yzd-nbjv-qyx 83 It Is Likely That Your Loss Should be a Likelihood Mark Hamilton, Evan Shelhamer and William Freeman https://mit.zoom.us/j/98388816271?pwd=cTZscGVtNzhzTXUzUGxTcUtpVkh3dz09 udl 84 Self-Adaptive Training: beyond Empirical Risk Minimization Huang, Chao Zhang and Hongyang Zhang https://meet.google.com/cek-gsff-ebv 85 BaCOUn: Bayesian Classifers with Out-of-Distribution Uncertainty Théo Guenais, Dimitris Vamvourellis, Yaniv Yacoby, Finale Doshi-Velez and Weiwei Pan https://us02web.zoom.us/j/84495842174?pwd=czNvaDkrMmZGRVgvdjlEczJxWUJwUT09 86 Bayesian active learning for production, a systematic study and a reusable library Parmida Atighehchian, Frédéric Branchaud-Charron and Alexandre Lacoste https://elementai.zoom.us/j/92111241672?pwd=NVg2V2Myd1lmKzFaWXlCa1dpNHZMUT09 87 Certainty as Supervision for Test-Time Adaptation Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen and Trevor Darrell https://berkeley.zoom.us/j/92105695905?pwd=cEk0T0U5Y0RDb2ZFSGZ2b09HTStIdz09 88 Robust Deep Reinforcement Learning through Adversarial Loss Tuomas Oikarinen, Tsui-Wei Weng and Luca Daniel https://mit.zoom.us/j/92724256949?pwd=K3JRdHhhVG1xV2tRbE81WXJQUGZsdz09 89 Cold Posteriors and Aleatoric Uncertainty Ben Adlam, Sam Smith and Jasper Snoek meet.google.com/ncb-ffxx-qgc 90 Ensemble Mean vs. Ensemble Variance: Which is a Better Uncertainty Metric for Incipient Disease Detection? Baihong Jin, Yingshui Tan, Xiangyu Yue, Yuxin Chen and Alberto Sangiovanni-Vincentelli https://berkeley.zoom.us/j/95691253740?pwd=SDFlR2hXYkRBQWFJZ3cydXlwRnEyZz09 91 Simplicity Bias and the Robustness of Neural Networks Harshay Shah, Kaustav Tamuly, Aditi Raghunathan, Prateek Jain and Praneeth Netrapalli https://meet.google.com/bdq-nckv-tyq 92 Exact posterior distributions of wide Bayesian neural networks Jiri Hron, Yasaman Bahri, Roman Novak, Jeffrey Pennington and Jascha Sohl-Dickstein https://meet.google.com/pvb-brkx-png 93 A Simulation-based Framework for Characterizing Predictive Distributions for Deep Learning Jessica Ai, Beliz Gokkaya, Ilknur Kaynar Kabul, Erik Meijer, Audrey Flower, Ehsan Emamjomeh-Zadeh, Hannah Li, Li Chen, Neamah Hussein, Ousmane Dia and Sevi Baltaoglu https://us04web.zoom.us/j/72581014579?pwd=MFlJeXRKZTBJdWw4eDNoeTJEcGZSUT09 94 Structured Weight Priors for Convolutional Neural Networks Tim Pearce, Andrew Y.K. Foong and Alexandra Brintrup meet.google.com/rzg-vdpa-xir 95 Learning Generative Models from Classifier Uncertainties Siddharth Narayanaswamy and Brooks Paige https://meet.google.com/gjz-innd-iyt 96 DQI: A Guide to Benchmark Evaluation Swaroop Mishra, Anjana Arunkumar, Bhavdeep Sachdeva, Chris Bryan and Chitta Baral https://asu.zoom.us/j/95489346758 97 Can Your AI Differentiate Cats from Covid-19? Sample Efficient Uncertainty Estimation for Deep Learning Safety Ankur Mallick, Chaitanya Dwivedi, Bhavya Kailkhura, Gauri Joshi and T. Yong-Jin Han https://cmu.zoom.us/j/95091389695?pwd=K2RXbzZVSitzRVZadHB4eHZaZmlDZz09 98 Transferable Adversarial Examples for Atari 2600 Games Damian Stachura and Michał Zając https://meet.google.com/jwf-nqyd-fty 99 Robustness to Distribution Shifts using Multiple Environments Anders Andreassen, Rebecca Roelofs and Behnam Neyshabur [ protected link dropped ] 100 Our Evaluation Metric Needs an Update to Encourage Generalization Swaroop Mishra, Anjana Arunkumar and Chris Bryan https://asu.zoom.us/j/92127915576 101 Harder or Different? A Closer Look at Distribution Shift in Dataset Reproduction Shangyun Lu, Bradley Nott, Aaron Olson, Alberto Todeschini, Puya Vahabi, Yair Carmon and Ludwig Schmidt https://stanford.zoom.us/j/98068202076?pwd=LytZdjg5bTR5eldVcFFHQVpSaHRzZz09 102 Empirical Scoring Rule Decomposition in Deep Learning Tony Duan https://teams.microsoft.com/l/meetup-join/19%3ameeting_NTBkNzE2OTYtODQwOS00MjI0LTgyNjQtY2Q3YWRhN2M1ZWNj%40thread.v2/0?context=%7b%22Tid%22%3a%2272f988bf-86f1-41af-91ab-2d7cd011db47%22%2c%22Oid%22%3a%2233328e52-be2d-485a-a0c5-7a8d97aaa4e2%22%7d 103 Failure Prediction by Confidence Estimation of Uncertainty-Aware Dirichlet Networks Theodoros Tsiligkaridis https://mit.zoom.us/j/95278652551?pwd=WHNwNEsxUUVOd0FSaUVHbkJaNHp5QT09 140350 104 RayS: A Ray Searching Method for Hard-label Adversarial Attack Jinghui Chen and Quanquan Gu https://ucla.zoom.us/j/92290062678?pwd=MThrUVVDSnlEU2FNWG10Yk1CcnRlZz09 987689 105 Joint Energy-Based Models for Semi-Supervised Classification Stephen Zhao, Joern-Henrik Jacobsen and Will Grathwohl https://us02web.zoom.us/j/88327646584?pwd=ZkxBcDNRMU9WVUt0ODJ6SDhoYk5zZz09 231335 106 Single Shot MC Dropout Approximation Kai Brach, Beate Sick and Oliver Duerr https://us02web.zoom.us/j/89892013425?pwd=VDZQN2JLSy9hTVg3VGpqOFhBdVJiZz09 1yqhv2 107 Reliable Uncertainties for Bayesian Neural Networks using Alpha-divergences Hector Javier Hortua, Luigi Malago and Riccardo Volpi [ protected link dropped ] 8a3QWf 108 Evaluating Uncertainty Estimation Methods on 3D Semantic Segmentation of Point Clouds Swaroop Bhandary, Nico Hochgeschwender, Paul Plöger and Matias Valdenegro-Toro [ protected link dropped ] 109 Improving predictions of Bayesian neural networks via local linearization Alexander Immer, Maciej Korzepa and Matthias Bauer https://ethz.zoom.us/j/98684846685?pwd=aVJFS3VNdSsyMGVzOFQvbW5qSisrZz09 110 URSABench: Comprehensive Benchmarking of Approximate Bayesian Inference Methods for Deep Neural Networks Meet Vadera, Adam Cobb, Brian Jalaian and Benjamin Marlin https://umass-amherst.zoom.us/j/99916090662?pwd=N1E1Mm9LcEQ4bmhjNWlWdTlyU3BGZz09 111 Richness of Training Data Does Not Suffice: Robustness of Neural Networks Requires Richness of Hidden-Layer Activations Kamil Nar and Shankar Sastry https://berkeley.zoom.us/j/98295724751?pwd=cW5QQ2EzL2RaUW15OWs3T3JrTjk2Zz09 112 Robust Classification under Class-Dependent Domain Shift Tigran Galstyan, Hrant Khachatrian, Greg Ver Steeg and Aram Galstyan https://us02web.zoom.us/j/82576216178?pwd=RzVGQlZUNE95S1NrRktaRk5WTXMrdz09 736349 113 You Need Only Uncertain Answers: Data Efficient Multilingual Question Answering Zhihao Lyu, Danier Duolikun, Bowei Dai, Yuan Yao, Pasquale Minervini, Tim Z. Xiao and Yarin Gal https://us02web.zoom.us/j/88327646584?pwd=ZkxBcDNRMU9WVUt0ODJ6SDhoYk5zZz09 231335 114 Diverse Ensembles Improve Calibration Asa Cooper Stickland and Iain Murray [ protected link dropped ] 115 Exploring the Uncertainty Properties of Neural Networks’ Implicit Priors in the Infinite-Width Limit Ben Adlam, Jaehoon Lee, Lechao Xiao, Jeffrey Pennington and Jasper Snoek meet.google.com/agb-migr-qdt 116 Provable Robust Learning Based on Transformation-Specific Smoothing Linyi Li, Maurice Weber, Xiaojun Xu, Luka Rimanic, Shuang Yang, Tao Xie, Ce Zhang and Bo Li https://ethz.zoom.us/j/98321715637?pwd=Zk1hS0J1NUlteWYxUmtUYVlXb2hOdz09 117 Simple and Effective VAE Training with Calibrated Decoders Oleh Rybkin, Kostas Daniilidis and Sergey Levine meet.google.com/hvr-sdbx-syc 118 Amortized Conditional Normalized Maximum Likelihood Aurick Zhou and Sergey Levine https://berkeley.zoom.us/j/4913207255?pwd=VGN5bXRJQi9HRXJEbFp6SjZ3a29idz09 119 Neural Networks with Recurrent Generative Feedback Yujia Huang, James Gornet, Sihui Dai, Zhiding Yu, Tan Nguyen, Doris Y. Tsao and Anima Anandkumar https://us02web.zoom.us/j/6830799787?pwd=NGU3VFRvWTdhbTNiK1YzVHgzMEs2QT09 9appDa 120 Robustness of Latent Representations of Variational Autoencoders Andrea Karlova https://meet.google.com/hmk-kory-kba 121 Learning approximate invariance requires far fewer data Jean Michel Amath Sarr, Alassane Bah and Christophe Cambier https://us04web.zoom.us/j/75476163865?pwd=Z3RES1dMWFVWZkE4YjM2RzRLWUJ6Zz09 122 MCU-Net: A framework towards uncertainty representations for decision support system patient referrals in healthcare contexts Nabeel Seedat [ protected link dropped ] 123 Uncertainty-sensitive Learning and Planning with Ensembles Piotr Milos, Lukasz Kucinski, Konrad Czechowski, Piotr Kozakowski and Maciej Klimek https://us02web.zoom.us/j/89300913614?pwd=cVJjdFRpaDFZRVRMQXBsSzZzOGVSQT09 124 Uncertainty in Multi-Interaction Trajectory Reconstruction Vasileios Karavias, Ben Day and Pietro Lio https://meet.google.com/uqx-qudh-ish 125 Unsupervised Domain Adaptation in the Absence of Source Data Roshni Sahoo, Divya Shanmugam and John Guttag https://mit.zoom.us/j/94674840454?pwd=L2NMaHBsaTRYZnE4TjZydmJERCtqUT09 126 A benchmark study on reliable molecular supervised learning via Bayesian learning Doyeong Hwang, Grace Lee, Hanseok Jo, Seyoul Yoon and Seongok Ryu https://meet.google.com/hiy-bbdm-qxo 127 On the Power of Oblivious Poisoning Attacks Samuel Deng, Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mahmoody and Abhradeep Thakurta https://virginia.zoom.us/j/97053762096 128 Training Deep Neural Networks with Class-level Semantics for Explicable Classification Alberto Olmo, Sailik Sengupta and Subbarao Kambhampati https://asu.zoom.us/j/93814557218?pwd=K1hjc3NqWXlLQm1pN0tHTk1rdUNJZz09 129 End-to-end Robustness for Sensing-Reasoning Machine Learning Pipelines Zhuolin Yang, Zhikuan Zhao, Hengzhi Pei, Boxin Wang, Bojan Karlas, Ji Liu, Heng Guo, Bo Li and Ce Zhang https://illinois.zoom.us/j/2530610336?pwd=WXVZYlkreDYyUjRNdFh0Qk1IMURjUT09 password 633681 130 Deep Robust Classification under Domain Shift with Conservative Uncertainty Estimation Haoxuan Wang, Anqi Liu, Yisong Yue and Anima Anandkumar https://caltech.zoom.us/j/92474618222?pwd=WS91UjhVY1p4WThHQjRRYU5LMnpJQT09 131 On Separability of Self-Supervised Representations Vikash Sehwag, Mung Chiang and Prateek Mittal https://princeton.zoom.us/j/3871410688?pwd=RldBOUNwMnhFU3d4ZGpRSVR2YWJSdz09 132 Dropout Strikes Back: Improved Uncertainty Estimation via Diversity Sampling Evgenii Tsymbalov, Kirill Fedyanin and Maxim Panov [ protected link dropped ] 133 Predictive Uncertainty for Probabilistic Novelty Detection in Text Classification Jordy Van Landeghem, Matthew Blaschko, Bertrand Anckaert and Marie-Francine Moens https://us02web.zoom.us/j/86332883100?pwd=TVd3UytVTkF1R1V1RlNpdTZqbWlaZz09 134 Maximizing the Representation Gap between In-domain & OOD examples Jay Nandy, Wynne Hsu and Mong Li Lee https://nus-sg.zoom.us/j/99627439405?pwd=WFlkNzhndlo3TTU5YytyenduN0NWQT09 135 Regional Image Perturbation Reduces Lp Norms of Adversarial Examples While Maintaining Model-to-model Transferability Utku Ozbulak, Jonathan Peck, Wesley De Neve, Bart Goossens, Yvan Saeys and Arnout Van Messem [ protected link dropped ] 135 136 Lookahead Adversarial Semantic Segmentation Hadi Jamali-Rad, Attila Szabo and Matteo Presutto meet.google.com/jgc-nfyp-nsn 137 Neural Network Calibration for Medical Imaging Classification Using DCA Regularization Gongbo Liang, Yu Zhang and Nathan Jacobs https://uky.zoom.us/j/93398011263?pwd=RUlBYzU3N1N4ekdzWEdZZ0czSnJpQT09 138 Deep k-NN Defense Against Clean-Label Data Poisoning Attacks Neehar Peri, Neal Gupta, Ronny Huang, Chen Zhu, Liam Fowl, Soheil Feizi, Tom Goldstein and John Dickerson https://umd.zoom.us/j/93698080878?pwd=S2VjZjAzZHB3RXlSUWtiRWh6K1N0dz09 139 Amortised Variational Inference for Hierarchical Mixture Models Javier Antoran, Jiayu Yao, Weiwei Pan, Finale Doshi-Velez and Jose Miguel Hernandez-Lobato https://harvard.zoom.us/j/95984006384?pwd=N09BSlRGWWR5T1czNE5vQjBGbm9jdz09 140 Benchmarking Search Methods for Generating NLP Adversarial Examples Jin Yong Yoo, John X. Morris, Eli Lifland and Yanjun Qi https://virginia.zoom.us/j/97965655398?pwd=aWd2ZW1EK2l1Y2wzL3ZMVGF0QW1rQT09 |
🔗 |
Fri 10:00 a.m. - 10:30 a.m.
|
Coffee Break
|
🔗 |
Fri 10:30 a.m. - 11:00 a.m.
|
Keynote #2 Finale Doshi-Velez
(
Keynote
)
link »
SlidesLive Video » http://slideslive.com/38930573 |
Finale Doshi-Velez 🔗 |
Fri 11:00 a.m. - 11:30 a.m.
|
Keynote #3 Percy Liang
(
Keynote
)
link »
SlidesLive Video » http://slideslive.com/38930574 |
Percy Liang 🔗 |
Fri 11:30 a.m. - 12:30 p.m.
|
Panel Discussion
(
Panel
)
link »
https://us02web.zoom.us/s/85455699146?pwd=REw5QkhTNk5MV3cxUVp5VnNzRWxhZz09 |
🔗 |
Fri 12:30 p.m. - 1:30 p.m.
|
Lunch Break
|
🔗 |
Fri 1:30 p.m. - 2:00 p.m.
|
Keynote #4 Raquel Urtasun
(
Keynote
)
link »
SlidesLive Video » http://slideslive.com/38930575 |
Raquel Urtasun 🔗 |
Fri 2:00 p.m. - 2:10 p.m.
|
Contributed Talk 1: Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks
(
Presentation
)
link »
SlidesLive Video » http://slideslive.com/38930576 |
David Stutz 🔗 |
Fri 2:10 p.m. - 2:20 p.m.
|
Contributed Talk 2: Improving robustness against common corruptions by covariate shift adaptation
(
Presentation
)
link »
SlidesLive Video » http://slideslive.com/38930577 |
Steffen Schneider 🔗 |
Fri 2:20 p.m. - 2:30 p.m.
|
Contributed Talk 3: A Unified View of Label Shift Estimation
(
Presentation
)
link »
SlidesLive Video » https://slideslive.com/38930578 |
Saurabh Garg 🔗 |
Fri 2:30 p.m. - 3:00 p.m.
|
Keynote #5 Justin Gilmer
(
Keynote
)
link »
SlidesLive Video » https://slideslive.com/38930579 |
Justin Gilmer 🔗 |
Fri 3:00 p.m. - 3:30 p.m.
|
Coffee Break
|
🔗 |
Fri 3:30 p.m. - 3:40 p.m.
|
Contributed Talk 4: A Benchmark of Medical Out of Distribution Detection
(
Presentation
)
link »
SlidesLive Video » https://slideslive.com/38930580 |
Joseph Paul Cohen 🔗 |
Fri 3:40 p.m. - 3:50 p.m.
|
Contributed Talk 5: Neural Ensemble Search for Performant and Calibrated Predictions
(
Presentation
)
link »
SlidesLive Video » https://slideslive.com/38930582 |
Sheheryar Zaidi 🔗 |
Fri 3:50 p.m. - 4:00 p.m.
|
Contributed Talk 6: Bayesian model averaging is suboptimal for generalization under model misspecification
(
Presentation
)
link »
SlidesLive Video » https://slideslive.com/38930581 |
Andres Arrendondo 🔗 |