Timezone: »

Uncertainty and Robustness in Deep Learning Workshop (UDL)
Sharon Yixuan Li · Balaji Lakshminarayanan · Dan Hendrycks · Thomas Dietterich · Jasper Snoek

Fri Jul 17 07:30 AM -- 04:00 PM (PDT) @ None
Event URL: https://sites.google.com/view/udlworkshop2020/home »

There has been growing interest in rectifying deep neural network instabilities. Challenges arise when models receive samples drawn from outside the training distribution. For example, a neural network tasked with classifying handwritten digits may assign high confidence predictions to cat images. Anomalies are frequently encountered when deploying ML models in the real world. Well-calibrated predictive uncertainty estimates are indispensable for many machine learning applications, such as self-driving vehicles and medical diagnosis systems. Generalization to unseen and worst-case inputs is also essential for robustness to distributional shift. In order to have ML models reliably predict in open environment, we must deepen technical understanding in the emerging areas of: (1) learning algorithms that can detect changes in data distribution (e.g. out-of-distribution examples); (2) mechanisms to estimate and calibrate confidence produced by neural networks in typical and unforeseen scenarios; (3) methods to improve out-of-distribution generalization, including generalization to temporal, geographical, hardware, adversarial, and image-quality changes; (4) benchmark datasets and protocols for evaluating model performance under distribution shift; and (5) key applications of robust and uncertainty-aware deep learning (e.g., computer vision, robotics, self-driving vehicles, medical imaging) as well as broader machine learning tasks.

This workshop will bring together researchers and practitioners from the machine learning communities, and highlight recent work that contributes to addressing these challenges. Our agenda will feature contributed papers with invited speakers. Through the workshop we hope to help identify fundamentally important directions on robust and reliable deep learning, and foster future collaborations.

Fri 7:30 a.m. - 7:40 a.m.
 link »   


Sharon Yixuan Li
Fri 7:40 a.m. - 8:10 a.m.
 link »   


Matthias Hein
Fri 8:10 a.m. - 8:15 a.m.
 link »   


Zhisheng Xiao
Fri 8:15 a.m. - 8:20 a.m.
 link »   


Yao-Yuan Yang
Fri 8:20 a.m. - 8:25 a.m.
 link »   


Javier Antorán, James Allingham
Fri 8:25 a.m. - 8:30 a.m.
 link »   


Jackson Wang
Fri 8:30 a.m. - 8:35 a.m.
 link »   


Luis Oala
Fri 8:35 a.m. - 8:40 a.m.
 link »   


Jishnu Mukhoti
Fri 8:40 a.m. - 8:45 a.m.
 link »   


Francesco Croce
Fri 8:45 a.m. - 8:50 a.m.
 link »   


Adam Oberman
Fri 9:00 a.m. - 10:00 a.m.

1 Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks David Stutz, Matthias Hein and Bernt Schiele https://zoom.us/j/99678534052

2 Improving robustness against common corruptions by covariate shift adaptation Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel and Matthias Bethge https://meet.google.com/ojt-atoh-wup

3 A Unified View of Label Shift Estimation Saurabh Garg, Yifan Wu, Sivaraman Balakrishnan and Zachary Lipton https://cmu.zoom.us/j/94831507038?pwd=RVVyVy96YXNiaGtjMUxyZUJ5dVcvQT09

4 A Benchmark of Medical Out of Distribution Detection Tianshi Cao, David Yu-Tung Hui, Chin-Wei Huang and Joseph Paul Cohen https://meet.google.com/jxk-paui-gao

5 Neural Ensemble Search for Performant and Calibrated Predictions Sheheryar Zaidi, Arber Zela, Thomas Elsken, Chris Holmes, Frank Hutter and Yee Whye Teh https://meet.google.com/uxs-uuxr-uwo

6 Bayesian model averaging is suboptimal for generalization under model misspecification Andres Masegosa https://meet.google.com/mpf-chva-pgy

7 Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder Zhisheng Xiao, Qing Yan and Yali Amit https://uchicago.zoom.us/j/98748547880?pwd=Q3Y0dUVPUFphbGY4NmNJK2hwZndWUT09 Password: 370663

8 A Closer Look at Accuracy vs. Robustness Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Ruslan Salakhutdinov and Kamalika Chaudhuri https://ucsd.zoom.us/j/92778085557?pwd=STJVVWMza0RKSFZsOUVIS0h6bVl5QT09

9 Depth Uncertainty in Neural Networks Javier Antorán, James Urquhart Allingham and José Miguel Hernández-Lobato https://us02web.zoom.us/j/5419103161?pwd=eXdqWURLc3o4SktwQWZOU2pZNkliQT09

10 Few-shot Out-of-Distribution Detection Kuan-Chieh Wang, Paul Vicol, Eleni Triantafillou and Richard Zemel https://vectorinstitute.zoom.us/j/97199418667?pwd=NjBWUXhzZUwrOHFBMkxSZEd2MDh3Zz09

11 Detecting Failure Modes in Image Reconstructions with Interval Neural Network Uncertainty Luis Oala, Cosmas Heiß, Jan Macdonald, Maximilian März, Gitta Kutyniok and Wojciech Samek https://us02web.zoom.us/j/88251013741?pwd=TzlOUlJXeExzUVdSUDE5RXFkTytWZz09

12 On using Focal Loss for Neural Network Calibration Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip Torr and Puneet Dokania https://meet.google.com/ehz-rzxf-xeu

13 AutoAttack: reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks Francesco Croce and Matthias Hein https://zoom.us/j/98193746046?pwd=RWNYVEJMQ1FzRGIwUDgrd1c1NmtVdz09

14 Calibrated Top-1 Uncertainty estimates for classification by score based models Adam Oberman, Chris Finlay, Alexander Iannantuono and Tiago Salvador https://mcgill.zoom.us/j/91308261627

15 Bayesian Deep Ensembles via the Neural Tangent Kernel Bobby He, Balaji Lakshminarayanan and Yee Whye Teh https://meet.google.com/exb-jkju-vbr

16 The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt and Justin Gilmer https://berkeley.zoom.us/j/4219480859?pwd=QkNIYmNCZjlobEUxQ2Q5TzR4Qm1QQT09

17 Measuring Robustness to Natural Distribution Shifts in Image Classification Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht and Ludwig Schmidt https://berkeley.zoom.us/j/97514838599?pwd=SXdiOSthMzNUV01QYU5lVElFTTM2dz09

18 Redundant features can hurt robustness to distribution shift Guillermo Ortiz-Jimenez, Apostolos Modas, Seyed-Mohsen Moosavi-Dezfooli and Pascal Frossard https://epfl.zoom.us/j/99757698661?pwd=NVc5OUVHenFYc2pXVlFYbk12U0F4dz09 548657

19 Scalable Training with Information Bottleneck Objectives Andreas Kirsch, Clare Lyle and Yarin Gal https://meet.google.com/mzn-pjuh-kia

20 How Does Early Stopping Help Generalization Against Label Noise? Hwanjun Song Hwanjun, Minseok Kim, Dongmin Park and Jae-Gil Lee https://zoom.us/j/3881124678?pwd=QkVrSCt1R243aW5td2NqNVNQY3paUT09

21 Soft Labeling Affects Out-of-Distribution Detection of Deep Neural Networks Doyup Lee and Yeongjae Cheon https://zoom.us/j/8641422727?pwd=RjlKakwzSW9JTHBjZGdST0YydHBOdz09 0deFHw

22 Learning Robust Representations with Score Invariant Learning Daksh Idnani and Jonathan Kao https://ucla.zoom.us/j/98883773408?pwd=Y3M2WmxNTW1HOS9hMlBZYmJJSHZ0dz09

23 PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction Sangdon Park, Osbert Bastani, Nikolai Matni and Insup Lee https://us04web.zoom.us/j/76965715700?pwd=U2tNWVBMd0RKMHcrNGlTa0VMQkY3dz09

24 Learned Uncertainty-Aware (LUNA) Bases for Bayesian Regression using Multi-Headed Auxiliary Networks Sujay Thakur, Cooper Lorsung, Yaniv Yacoby, Finale Doshi-Velez and Weiwei Pan https://harvard.zoom.us/j/96201359957?pwd=N3Q3RVJJZ0R0TmRuejYxTDRDMjAxUT09

25 Domain Generalization using Causal Matching Divyat Mahajan, Shruti Tople and Amit Sharma https://us02web.zoom.us/j/87167858273?pwd=V09XWTdjMzVSTHplcnJ6dCtpUWphUT09

26 Hydra: Preserving Ensemble Diversity for Model Distillation Linh Tran, Bastiaan S. Veeling, Kevin Roth, Jakub Swiatkowski, Joshua V. Dillon, Stephan Mandt, Jasper Snoek, Tim Salimans, Sebastian Nowozin and Rodolphe Jenatton meet.google.com/vqj-zckz-hbr

27 Predicting with High Correlation Features Devansh Arpit, Caiming Xiong and Richard Socher https://us04web.zoom.us/j/3643443782

28 Consistency Regularization for Certified Robustness of Smoothed Classifiers Jongheon Jeong and Jinwoo Shin https://us02web.zoom.us/j/85431063603?pwd=MGJ4YzR4Q2FXSWM4c3c3NURiR3NqZz09

29 DIBS: Diversity inducing Information Bottleneck in Model Ensembles Samarth Sinha, Homanga Bharadhwaj, Anirudh Goyal, Hugo Larochelle, Animesh Garg and Florian Shkurti https://utoronto.zoom.us/j/6995996022?pwd=UUIxNUNnQThLVmhLV0h5RDVJRG52dz09

30 On the relationship between class selectivity, dimensionality, and robustness Matthew Leavitt and Ari Morcos https://fb.zoom.us/j/98555977785 949102

31 Robust Variational Autoencoder for Tabular Data with β Divergence Haleh Aydore, Sergul Aydore, Richard Leahy and Anand Joshi https://usc.zoom.us/j/92860013817?pwd=aG5YekNUcFBIalV4TVNJYkhyb1UzZz09 906642

32 Uncertainty in Structured Prediction Andrey Malinin and Mark Gales https://yandex.zoom.us/j/98605165946?pwd=RkJ4RTVmYmt6eU1CcWU5RGgvNEMzdz09

33 Understanding and Improving Fast Adversarial Training Maksym Andriushchenko and Nicolas Flammarion https://epfl.zoom.us/j/98065886762?pwd=dTVFMUdFZllIZFVEMllDbkdxL2QvZz09

34 Predictive Complexity Priors Eric Nalisnick, Jonathan Gordon and Jose Miguel Hernandez Lobato https://meet.google.com/jyk-qhvg-kek

35 Provable Worst Case Guarantees for the Detection of Out-of-Distribution Data Julian Bitterwolf, Alexander Meinke and Matthias Hein https://zoom.us/j/94329789686?pwd=WHlXakluajZ1bkRkRUl2N0YyMEgrQT09

36 Ensemble Distribution Distillation via Regression Prior Networks Andrey Malinin, Sergey Chervontsev, Ivan Provilkov and Mark Gales meet.google.com/zbn-vwtv-eda

37 Generalizing to unseen domains via distribution matching Isabela Albuquerque, Joao Monteiro, Mohammad Darvishi, Tiago Falk and Ioannis Mitliagkas https://zoom.us/j/98309028439?pwd=dzZXbS9GUTduMk5mM3F4dWZqc2lLQT09 2U0Qsc

38 GAN-mixup: Augmenting Across Data Manifolds for Improved Robustness Jy-Yong Sohn, Kangwook Lee, Jaekyun Moon and Dimitris Papailiopoulos https://us02web.zoom.us/j/84163398388?pwd=MGFXWm9VZGhDZ01mMXFCMkNJckdRdz09 240878

39 Improving out-of-distribution generalization via multi-task self-supervised pretraining Isabela Albuquerque, Nikhil Naik, Junnan Li, Nitish Shirish Keskar and Richard Socher https://zoom.us/j/96392222189?pwd=bVg5am4vb0FXYVZMNEtPWER4SURyUT09 6byCUy

40 Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks Shreyas Padhy, Zachary Nado, Jie Ren, Jeremiah Liu, Jasper Snoek and Balaji Lakshminarayanan https://meet.google.com/cym-bnec-mbb

41 Tilted Empirical Risk Minimization Tian Li, Ahmad Beirami, Maziar Sanjabi and Virginia Smith https://cmu.zoom.us/j/96622035970?pwd=M21WMTllV0tuWkpMaUlHMDlBWmpFdz09

42 Towards Robust Classification with Deep Generative Forests Alvaro Henrique Chaim Correia, Robert Peharz and Cassio de Campos https://zoom.us/j/98359572057?pwd=VDEzUHE5Q1JkVFBoK1RQUTJTTUlRQT09

43 Riemannian Continuous Normalizing Flows Emile Mathieu https://meet.google.com/vts-axeh-doo

44 Improving Calibration of BatchEnsemble with Data Augmentation Yeming Wen, Ghassen Jerfel, Rafael Muller, Michael Dusenberry, Jasper Snoek, Balaji Lakshminarayanan and Dustin Tran https://us04web.zoom.us/j/79191629966?pwd=eVNEZ2NRVGY4NEc0Uk1DK2t1Ykh6QT09.

45 Environment Inference for Invariant Learning Elliot Creager, Jörn-Henrik Jacobsen and Richard Zemel https://vectorinstitute.zoom.us/j/98611353513?pwd=Mk5xa2VqLzVBV0JzdjZINXNlWXJMdz09 470387

46 Nonlinear Gradient Estimation for Query Efficient Blackbox Attack Huichen Li, Linyi Li, Xiaojun Xu, Xiaolu Zhang, Shuang Yang and Bo Li https://illinois.zoom.us/j/92637811021?pwd=WldhQXRVaENIWXFNQjdrdXhmcG1UQT09 NLBA

47 ImageNet performance correlates with pose estimation robustness and generalization on out-of-domain data Alexander Mathis, Thomas Biasi, Mert Yüksegönül, Byron Rogers, Matthias Bethge and Mackenzie Mathis https://harvard.zoom.us/j/96654497889?pwd=Z2NrL05lQlozZndsQ25jV0JQdkRtZz09

48 CRUDE: Calibrating Regression Uncertainty Distributions Empirically Eric Zelikman, Christopher Healy, Sharon Zhou and Anand Avati https://stanford.zoom.us/j/92845303885?pwd=Kzd6d2tjMGdXL0VkK01JM09jYkVqUT09 519357

49 Positive-Unlabeled Learning with Arbitrarily Non-Representative Labeled Data Zayd Hammoudeh and Daniel Lowd https://uoregon.zoom.us/j/99760928750?pwd=dmVCUlI3WUlxNldDRGVqQmQrSDVFQT09 359623

50 Probabilistic Robustness Estimates for Deep Neural Networks Nicolas Couellan https://zoom.us/j/92492909798?pwd=NUo4QzlPM0JLVmZCTW9HOGdpZ3lidz09

51 Estimating Risk and Uncertainty in Deep Reinforcement Learning William Clements, Bastien Van Delft, Benoît-Marie Robaglia, Reda Bahi Slaoui and Sébastien Toth https://us02web.zoom.us/j/84543888187?pwd=cVdIS1ZjQUVzaTNkR0lRT2lTVzNxZz09

52 An Empirical Study of Invariant Risk Minimization Yo Joong Choe, Jiyeon Ham and Kyubyong Park https://cmu.zoom.us/j/97184088310?pwd=MGJ0bEYzQXg3RVhHSkxTTWY2RnBHdz09

53 Information-Bottleneck under Mean Field Initialization Vinayak Abrol and Jared Tanner https://zoom.us/j/99737001164?pwd=ZnNLdXI4SkFtTmlkdVYySzQvKzc5Zz09

54 Adaptive Risk Minimization: A Meta-Learning Approach for Tackling Group Shift Marvin Zhang, Henrik Marklund, Abhishek Gupta, Sergey Levine and Chelsea Finn https://berkeley.zoom.us/j/96584472302?pwd=TXlPWHFzeHhVTzROYnB4KzJVSVJhUT09

55 Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift Zachary Nado, Shreyas Padhy, D. Sculley, Alexander D’amour, Balaji Lakshminarayanan and Jasper Snoek https://meet.google.com/zxr-qijn-xkx

56 Failures of Variational Autoencoders and their Effects on Downstream Tasks Yaniv Yacoby, Weiwei Pan and Finale Doshi-Velez https://harvard.zoom.us/j/98541353638?pwd=QUovTjVKYm4rM1NSamw1VUJ3eVI2Zz09

57 Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness Jeremiah Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax-Weiss and Balaji Lakshminarayanan https://zoom.us/j/95562904311?pwd=QUxIbzJGL0dCcG5QcDd2b011OGppZz09

58 Bayesian BERT for Trustful Hate Speech Detection Kristian Miok, Blaz Skrlj, Daniela Zaharie and Marko Robnik-Sikonja https://zoom.us/j/3443455903?pwd=UUwzYUlrYUs4YXY4RHpTRmZZcS9xdz09 12341234

59 On the Role of Dataset Quality and Heterogeneity in Model Confidence Yuan Zhao, Jiasi Chen and Samet Oymak https://us04web.zoom.us/j/76373416589?pwd=L09lNlBDUXcvNHZ6MjV3TTJUQzUxQT0

60 Certified Adversarial Robustness via Randomized Smoothing: a Case Study for Laplace Noises Jiaye Teng, Guanghe Lee and Yang Yuan https://zoom.us/j/94122377310?pwd=SGZoRXgxQ1A0U1dEYkVlZndnMnY3UT09 password 3QAa4p

61 QUEST for MEDISYN: Quasi-norm based Uncertainty ESTimation for MEDical Image SYNthesis Uddeshya Upadhyay, Viswanath P. Sudarshan and Suyash P. Awate https://monash.zoom.us/j/4586078207?pwd=OGlzeHh0bVovM3lkNCttRW40SFNMZz09 quest

62 On uncertainty estimation in active learning for image segmentation Bo Li and Tommy Sonne Alstrøm https://dtudk.zoom.us/j/65926847472?pwd=NGU0cXVlMy9oeGQ3dmp5WlFCcWgvUT09

63 Bayesian Autoencoders: Analysing and Fixing the Bernoulli likelihood for Out-of-Distribution Detection Bang Xiang Yong, Tim Pearce and Alexandra Brintrup meet.google.com/uxr-hzin-coj

64 A Comparison of Bayesian Deep Learning for Out of Distribution Detection and Uncertainty Estimation John Mitros, Arjun Pakrashi and Brian Mac Namee https://meet.google.com/puu-yysf-owt?hs=122&authuser=0

65 Practical Bayesian Neural Networks via Adaptive Subgradient Optimization Methods Samuel Kessler, Arnold Salas, Vincent Tan Weng Choon, Stefan Zohren and Stephen Roberts https://us02web.zoom.us/j/81195534206

66 Bayesian Few-Shot Classification with One-vs-Each Pólya-Gamma Augmented Gaussian Processes Jake Snell and Richard Zemel https://vectorinstitute.zoom.us/j/99320884951?pwd=aEdhbE8wUWNIS1ZuZkc4ZHFKVGNWUT09

67 Characteristics of Monte Carlo Dropout in Wide Neural Networks Joachim Sicking, Maram Akila, Tim Wirtz, Sebastian Houben and Asja Fischer https://us02web.zoom.us/j/88053290381?pwd=T2FaaXk1aUhWcS9iSFFhTW9WY1NTdz09

68 On Power Laws in Deep Ensembles Ekaterina Lobacheva, Nadezhda Chirkova, Maxim Kodryan and Dmitry Vetrov https://zoom.us/j/95153236006?pwd=bTJDM3M0NlBvelZXRFJHay9XZHROQT09

69 Outlier Detection through Null Space Analysis of Neural Networks Matthew Cook, Alina Zare and Paul Gader https://ufl.zoom.us/j/94556341852?pwd=MXRoVUw5M3BUMmFYcjlGOTBVdDhIQT09

70 In a forward direction: Analyzing distribution shifts in machine translation test sets over time Thomas Liao, Benjamin Recht and Ludwig Schmidt https://berkeley.zoom.us/j/98684240218?pwd=elZ3RVVOQllkdDZGTU1PZTY0dGZPUT09

71 Characterizing Adversarial Transferability via Gradient Orthogonality and Smoothness Zhuolin Yang, Linyi Li, Xiaojun Xu, Kaizhao Liang, Shiliang Zuo, Qian Chen, Benjamin Rubinstein, Ce Zhang and Bo Li https://illinois.zoom.us/j/95161238471?pwd=NHRrZ1c0ZlAwMVp6U2Vwd3U2OXk1dz09 926964

72 Untapped Potential of Data Augmentation: A Domain Generalization Viewpoint Vihari Piratla and Shiv Shankar https://meet.google.com/xya-jmcm-htd

73 Classifying Perturbation Types for Adversarial Robustness Against Multiple Threat Models Pratyush Maini, Xinyun Chen, Bo Li and Dawn Song https://berkeley.zoom.us/j/96355581753?pwd=djFrc0hMVmQ1RU5rMzNsbCtjbXVQdz09

74 A Critical Evaluation of Open-World Machine Learning Liwei Song, Vikash Sehwag, Arjun Nitin Bhagoji and Prateek Mittal https://princeton.zoom.us/j/6441846264?pwd=RWgzYjA2aWJVbnJvaVcvc2FQRm1HUT09

75 You won't believe you can learn CIFAR-10 with this: Another take on Information Bottleneck Objectives Andreas Kirsch, Clare Lyle and Yarin Gal https://meet.google.com/tes-egvb-nso

76 Principled Uncertainty Estimation for High Dimensional Data Pascal Notin, José Miguel Hernández-Lobato and Yarin Gal https://meet.google.com/xte-ywjb-paj

77 Rethink Autoencoders: Robust Manifold Learning Taihui Li, Rishabh Mehta, Zecheng Qian and Ju Sun https://umn.zoom.us/j/92216511031?pwd=UjN1K2IvRFVvaDlpVUlXZWFqMURZUT09 7zcn93

78 Continuous-Depth Bayesian Neural Networks Winnie Xu, Ricky T.Q. Chen and David Duvenaud https://utoronto.zoom.us/j/7553398989 755996

79 Robust Temporal Point Event Localization through Smoothing and Counting Julien Schroeter, Kirill Sidorov and David Marshall https://cardiff.zoom.us/j/96449530903?pwd=RjBtNDFmY0ZlT1EyMFJRZjdNdDh0UT09

80 Chi-square Information for Invariant Learning Prasanna Sattigeri, Soumya Ghosh and Samuel Hoffman https://us04web.zoom.us/j/77772482601?pwd=MXdDTDRUWmFJbktYcWVkT3NRRWUxdz09

81 Robust Out-of-distribution Detection via Informative Outlier Mining Jiefeng Chen, Sharon Li, Xi Wu, Yingyu Liang and Somesh Jha https://stanford.zoom.us/j/99582468298?pwd=V1VINTI1U1JJalRBaktTQnk4c0VvQT09

82 An Empirical Analysis of the Impact of Data Augmentation on Distillation Deepan Das, Haley Massa, Abhimanyu Kulkarni and Theodoros Rekatsinas https://meet.google.com/yzd-nbjv-qyx

83 It Is Likely That Your Loss Should be a Likelihood Mark Hamilton, Evan Shelhamer and William Freeman https://mit.zoom.us/j/98388816271?pwd=cTZscGVtNzhzTXUzUGxTcUtpVkh3dz09 udl

84 Self-Adaptive Training: beyond Empirical Risk Minimization Huang, Chao Zhang and Hongyang Zhang https://meet.google.com/cek-gsff-ebv

85 BaCOUn: Bayesian Classifers with Out-of-Distribution Uncertainty Théo Guenais, Dimitris Vamvourellis, Yaniv Yacoby, Finale Doshi-Velez and Weiwei Pan https://us02web.zoom.us/j/84495842174?pwd=czNvaDkrMmZGRVgvdjlEczJxWUJwUT09

86 Bayesian active learning for production, a systematic study and a reusable library Parmida Atighehchian, Frédéric Branchaud-Charron and Alexandre Lacoste https://elementai.zoom.us/j/92111241672?pwd=NVg2V2Myd1lmKzFaWXlCa1dpNHZMUT09

87 Certainty as Supervision for Test-Time Adaptation Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen and Trevor Darrell https://berkeley.zoom.us/j/92105695905?pwd=cEk0T0U5Y0RDb2ZFSGZ2b09HTStIdz09

88 Robust Deep Reinforcement Learning through Adversarial Loss Tuomas Oikarinen, Tsui-Wei Weng and Luca Daniel https://mit.zoom.us/j/92724256949?pwd=K3JRdHhhVG1xV2tRbE81WXJQUGZsdz09

89 Cold Posteriors and Aleatoric Uncertainty Ben Adlam, Sam Smith and Jasper Snoek meet.google.com/ncb-ffxx-qgc

90 Ensemble Mean vs. Ensemble Variance: Which is a Better Uncertainty Metric for Incipient Disease Detection? Baihong Jin, Yingshui Tan, Xiangyu Yue, Yuxin Chen and Alberto Sangiovanni-Vincentelli https://berkeley.zoom.us/j/95691253740?pwd=SDFlR2hXYkRBQWFJZ3cydXlwRnEyZz09

91 Simplicity Bias and the Robustness of Neural Networks Harshay Shah, Kaustav Tamuly, Aditi Raghunathan, Prateek Jain and Praneeth Netrapalli https://meet.google.com/bdq-nckv-tyq

92 Exact posterior distributions of wide Bayesian neural networks Jiri Hron, Yasaman Bahri, Roman Novak, Jeffrey Pennington and Jascha Sohl-Dickstein https://meet.google.com/pvb-brkx-png

93 A Simulation-based Framework for Characterizing Predictive Distributions for Deep Learning Jessica Ai, Beliz Gokkaya, Ilknur Kaynar Kabul, Erik Meijer, Audrey Flower, Ehsan Emamjomeh-Zadeh, Hannah Li, Li Chen, Neamah Hussein, Ousmane Dia and Sevi Baltaoglu https://us04web.zoom.us/j/72581014579?pwd=MFlJeXRKZTBJdWw4eDNoeTJEcGZSUT09

94 Structured Weight Priors for Convolutional Neural Networks Tim Pearce, Andrew Y.K. Foong and Alexandra Brintrup meet.google.com/rzg-vdpa-xir

95 Learning Generative Models from Classifier Uncertainties Siddharth Narayanaswamy and Brooks Paige https://meet.google.com/gjz-innd-iyt

96 DQI: A Guide to Benchmark Evaluation Swaroop Mishra, Anjana Arunkumar, Bhavdeep Sachdeva, Chris Bryan and Chitta Baral https://asu.zoom.us/j/95489346758

97 Can Your AI Differentiate Cats from Covid-19? Sample Efficient Uncertainty Estimation for Deep Learning Safety Ankur Mallick, Chaitanya Dwivedi, Bhavya Kailkhura, Gauri Joshi and T. Yong-Jin Han https://cmu.zoom.us/j/95091389695?pwd=K2RXbzZVSitzRVZadHB4eHZaZmlDZz09

98 Transferable Adversarial Examples for Atari 2600 Games Damian Stachura and Michał Zając https://meet.google.com/jwf-nqyd-fty

99 Robustness to Distribution Shifts using Multiple Environments Anders Andreassen, Rebecca Roelofs and Behnam Neyshabur https://zoom.us/j/91459170926?pwd=M25Sa01jWjZJeWlmNG5OSlV5Zmd6QT09

100 Our Evaluation Metric Needs an Update to Encourage Generalization Swaroop Mishra, Anjana Arunkumar and Chris Bryan https://asu.zoom.us/j/92127915576

101 Harder or Different? A Closer Look at Distribution Shift in Dataset Reproduction Shangyun Lu, Bradley Nott, Aaron Olson, Alberto Todeschini, Puya Vahabi, Yair Carmon and Ludwig Schmidt https://stanford.zoom.us/j/98068202076?pwd=LytZdjg5bTR5eldVcFFHQVpSaHRzZz09

102 Empirical Scoring Rule Decomposition in Deep Learning Tony Duan https://teams.microsoft.com/l/meetup-join/19%3ameeting_NTBkNzE2OTYtODQwOS00MjI0LTgyNjQtY2Q3YWRhN2M1ZWNj%40thread.v2/0?context=%7b%22Tid%22%3a%2272f988bf-86f1-41af-91ab-2d7cd011db47%22%2c%22Oid%22%3a%2233328e52-be2d-485a-a0c5-7a8d97aaa4e2%22%7d

103 Failure Prediction by Confidence Estimation of Uncertainty-Aware Dirichlet Networks Theodoros Tsiligkaridis https://mit.zoom.us/j/95278652551?pwd=WHNwNEsxUUVOd0FSaUVHbkJaNHp5QT09 140350

104 RayS: A Ray Searching Method for Hard-label Adversarial Attack Jinghui Chen and Quanquan Gu https://ucla.zoom.us/j/92290062678?pwd=MThrUVVDSnlEU2FNWG10Yk1CcnRlZz09 987689

105 Joint Energy-Based Models for Semi-Supervised Classification Stephen Zhao, Joern-Henrik Jacobsen and Will Grathwohl https://us02web.zoom.us/j/88327646584?pwd=ZkxBcDNRMU9WVUt0ODJ6SDhoYk5zZz09 231335

106 Single Shot MC Dropout Approximation Kai Brach, Beate Sick and Oliver Duerr https://us02web.zoom.us/j/89892013425?pwd=VDZQN2JLSy9hTVg3VGpqOFhBdVJiZz09 1yqhv2

107 Reliable Uncertainties for Bayesian Neural Networks using Alpha-divergences Hector Javier Hortua, Luigi Malago and Riccardo Volpi https://zoom.us/j/98397444333?pwd=b0JCTGhlOEFWTTh0Yld6akpDeXFzdz09 8a3QWf

108 Evaluating Uncertainty Estimation Methods on 3D Semantic Segmentation of Point Clouds Swaroop Bhandary, Nico Hochgeschwender, Paul Plöger and Matias Valdenegro-Toro https://zoom.us/j/99779971867?pwd=Qm9aQzJXYlJpV2ZwNmk4SU9BbFJQdz09

109 Improving predictions of Bayesian neural networks via local linearization Alexander Immer, Maciej Korzepa and Matthias Bauer https://ethz.zoom.us/j/98684846685?pwd=aVJFS3VNdSsyMGVzOFQvbW5qSisrZz09

110 URSABench: Comprehensive Benchmarking of Approximate Bayesian Inference Methods for Deep Neural Networks Meet Vadera, Adam Cobb, Brian Jalaian and Benjamin Marlin https://umass-amherst.zoom.us/j/99916090662?pwd=N1E1Mm9LcEQ4bmhjNWlWdTlyU3BGZz09

111 Richness of Training Data Does Not Suffice: Robustness of Neural Networks Requires Richness of Hidden-Layer Activations Kamil Nar and Shankar Sastry https://berkeley.zoom.us/j/98295724751?pwd=cW5QQ2EzL2RaUW15OWs3T3JrTjk2Zz09

112 Robust Classification under Class-Dependent Domain Shift Tigran Galstyan, Hrant Khachatrian, Greg Ver Steeg and Aram Galstyan https://us02web.zoom.us/j/82576216178?pwd=RzVGQlZUNE95S1NrRktaRk5WTXMrdz09 736349

113 You Need Only Uncertain Answers: Data Efficient Multilingual Question Answering Zhihao Lyu, Danier Duolikun, Bowei Dai, Yuan Yao, Pasquale Minervini, Tim Z. Xiao and Yarin Gal https://us02web.zoom.us/j/88327646584?pwd=ZkxBcDNRMU9WVUt0ODJ6SDhoYk5zZz09 231335

114 Diverse Ensembles Improve Calibration Asa Cooper Stickland and Iain Murray https://zoom.us/j/6892133874?pwd=QnBWS2FzcWkyQWYzdVJRaDRsbFYrUT09

115 Exploring the Uncertainty Properties of Neural Networks’ Implicit Priors in the Infinite-Width Limit Ben Adlam, Jaehoon Lee, Lechao Xiao, Jeffrey Pennington and Jasper Snoek meet.google.com/agb-migr-qdt

116 Provable Robust Learning Based on Transformation-Specific Smoothing Linyi Li, Maurice Weber, Xiaojun Xu, Luka Rimanic, Shuang Yang, Tao Xie, Ce Zhang and Bo Li https://ethz.zoom.us/j/98321715637?pwd=Zk1hS0J1NUlteWYxUmtUYVlXb2hOdz09

117 Simple and Effective VAE Training with Calibrated Decoders Oleh Rybkin, Kostas Daniilidis and Sergey Levine meet.google.com/hvr-sdbx-syc

118 Amortized Conditional Normalized Maximum Likelihood Aurick Zhou and Sergey Levine https://berkeley.zoom.us/j/4913207255?pwd=VGN5bXRJQi9HRXJEbFp6SjZ3a29idz09

119 Neural Networks with Recurrent Generative Feedback Yujia Huang, James Gornet, Sihui Dai, Zhiding Yu, Tan Nguyen, Doris Y. Tsao and Anima Anandkumar https://us02web.zoom.us/j/6830799787?pwd=NGU3VFRvWTdhbTNiK1YzVHgzMEs2QT09 9appDa

120 Robustness of Latent Representations of Variational Autoencoders Andrea Karlova https://meet.google.com/hmk-kory-kba

121 Learning approximate invariance requires far fewer data Jean Michel Amath Sarr, Alassane Bah and Christophe Cambier https://us04web.zoom.us/j/75476163865?pwd=Z3RES1dMWFVWZkE4YjM2RzRLWUJ6Zz09

122 MCU-Net: A framework towards uncertainty representations for decision support system patient referrals in healthcare contexts Nabeel Seedat https://zoom.us/j/91976554789?pwd=NC9xUDJDOUhZajkxUTV6VkxUK0kzQT09

123 Uncertainty-sensitive Learning and Planning with Ensembles Piotr Milos, Lukasz Kucinski, Konrad Czechowski, Piotr Kozakowski and Maciej Klimek https://us02web.zoom.us/j/89300913614?pwd=cVJjdFRpaDFZRVRMQXBsSzZzOGVSQT09

124 Uncertainty in Multi-Interaction Trajectory Reconstruction Vasileios Karavias, Ben Day and Pietro Lio https://meet.google.com/uqx-qudh-ish

125 Unsupervised Domain Adaptation in the Absence of Source Data Roshni Sahoo, Divya Shanmugam and John Guttag https://mit.zoom.us/j/94674840454?pwd=L2NMaHBsaTRYZnE4TjZydmJERCtqUT09

126 A benchmark study on reliable molecular supervised learning via Bayesian learning Doyeong Hwang, Grace Lee, Hanseok Jo, Seyoul Yoon and Seongok Ryu https://meet.google.com/hiy-bbdm-qxo

127 On the Power of Oblivious Poisoning Attacks Samuel Deng, Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mahmoody and Abhradeep Thakurta https://virginia.zoom.us/j/97053762096

128 Training Deep Neural Networks with Class-level Semantics for Explicable Classification Alberto Olmo, Sailik Sengupta and Subbarao Kambhampati https://asu.zoom.us/j/93814557218?pwd=K1hjc3NqWXlLQm1pN0tHTk1rdUNJZz09

129 End-to-end Robustness for Sensing-Reasoning Machine Learning Pipelines Zhuolin Yang, Zhikuan Zhao, Hengzhi Pei, Boxin Wang, Bojan Karlas, Ji Liu, Heng Guo, Bo Li and Ce Zhang https://illinois.zoom.us/j/2530610336?pwd=WXVZYlkreDYyUjRNdFh0Qk1IMURjUT09 password 633681

130 Deep Robust Classification under Domain Shift with Conservative Uncertainty Estimation Haoxuan Wang, Anqi Liu, Yisong Yue and Anima Anandkumar https://caltech.zoom.us/j/92474618222?pwd=WS91UjhVY1p4WThHQjRRYU5LMnpJQT09

131 On Separability of Self-Supervised Representations Vikash Sehwag, Mung Chiang and Prateek Mittal https://princeton.zoom.us/j/3871410688?pwd=RldBOUNwMnhFU3d4ZGpRSVR2YWJSdz09

132 Dropout Strikes Back: Improved Uncertainty Estimation via Diversity Sampling Evgenii Tsymbalov, Kirill Fedyanin and Maxim Panov https://zoom.us/j/99372767451?pwd=cHFaMjBIODRNZ0hUc1p2M0U5dWJTdz09

133 Predictive Uncertainty for Probabilistic Novelty Detection in Text Classification Jordy Van Landeghem, Matthew Blaschko, Bertrand Anckaert and Marie-Francine Moens https://us02web.zoom.us/j/86332883100?pwd=TVd3UytVTkF1R1V1RlNpdTZqbWlaZz09

134 Maximizing the Representation Gap between In-domain & OOD examples Jay Nandy, Wynne Hsu and Mong Li Lee https://nus-sg.zoom.us/j/99627439405?pwd=WFlkNzhndlo3TTU5YytyenduN0NWQT09

135 Regional Image Perturbation Reduces Lp Norms of Adversarial Examples While Maintaining Model-to-model Transferability Utku Ozbulak, Jonathan Peck, Wesley De Neve, Bart Goossens, Yvan Saeys and Arnout Van Messem https://zoom.us/j/7981481618?pwd=bFErdmJlUTA3bzRtTXN5U0hPa29YUT09 135

136 Lookahead Adversarial Semantic Segmentation Hadi Jamali-Rad, Attila Szabo and Matteo Presutto meet.google.com/jgc-nfyp-nsn

137 Neural Network Calibration for Medical Imaging Classification Using DCA Regularization Gongbo Liang, Yu Zhang and Nathan Jacobs https://uky.zoom.us/j/93398011263?pwd=RUlBYzU3N1N4ekdzWEdZZ0czSnJpQT09

138 Deep k-NN Defense Against Clean-Label Data Poisoning Attacks Neehar Peri, Neal Gupta, Ronny Huang, Chen Zhu, Liam Fowl, Soheil Feizi, Tom Goldstein and John Dickerson https://umd.zoom.us/j/93698080878?pwd=S2VjZjAzZHB3RXlSUWtiRWh6K1N0dz09

139 Amortised Variational Inference for Hierarchical Mixture Models Javier Antoran, Jiayu Yao, Weiwei Pan, Finale Doshi-Velez and Jose Miguel Hernandez-Lobato https://harvard.zoom.us/j/95984006384?pwd=N09BSlRGWWR5T1czNE5vQjBGbm9jdz09

140 Benchmarking Search Methods for Generating NLP Adversarial Examples Jin Yong Yoo, John X. Morris, Eli Lifland and Yanjun Qi https://virginia.zoom.us/j/97965655398?pwd=aWd2ZW1EK2l1Y2wzL3ZMVGF0QW1rQT09

Fri 10:00 a.m. - 10:30 a.m.
Coffee Break (Break)
Fri 10:30 a.m. - 11:00 a.m.
 link »   


Finale Doshi-Velez
Fri 11:00 a.m. - 11:30 a.m.
 link »   


Percy Liang
Fri 11:30 a.m. - 12:30 p.m.
 link »


Fri 12:30 p.m. - 1:30 p.m.
Lunch Break (Break)
Fri 1:30 p.m. - 2:00 p.m.
 link »   


Raquel Urtasun
Fri 2:00 p.m. - 2:10 p.m.
 link »   


David Stutz
Fri 2:10 p.m. - 2:20 p.m.
 link »   


Steffen Schneider
Fri 2:20 p.m. - 2:30 p.m.
 link »   


Saurabh Garg
Fri 2:30 p.m. - 3:00 p.m.
 link »   


Justin Gilmer
Fri 3:00 p.m. - 3:30 p.m.
Coffee Break (Break)
Fri 3:30 p.m. - 3:40 p.m.
 link »   


Joseph Paul Cohen
Fri 3:40 p.m. - 3:50 p.m.
 link »   


Sheheryar Zaidi
Fri 3:50 p.m. - 4:00 p.m.
 link »   


Andres Arrendondo

Author Information

Sharon Yixuan Li (Stanford University)

Sharon Y. Li is currently a postdoc researcher in the Computer Science department at Stanford, working with Chris Ré. She will be joining the Computer Sciences Department at University of Wisconsin Madison as an assistant professor, starting in Fall 2020. Previously, she completed her PhD from Cornell University in 2017, where she was advised by John E. Hopcroft. Her thesis committee members are Kilian Q. Weinberger and Thorsten Joachims. She has spent time at Google AI twice as an intern, and Facebook AI as a Research Scientist. She was named Forbes 30 Under 30 in Science in 2020. Her principal research interests are in the algorithmic foundations of deep learning and its applications. Her time in both academia and industry has shaped my view and approach in research. She is particularly excited about developing open-world machine learning methods that can reduce human supervision during training, and enhance reliability during deployment.

Balaji Lakshminarayanan (Google Brain)
Dan Hendrycks (UC Berkeley)
Tom Dietterich (Oregon State University)
Jasper Snoek (Google Brain)

More from the Same Authors