Timezone: »
There has been growing interest in rectifying deep neural network instabilities. Challenges arise when models receive samples drawn from outside the training distribution. For example, a neural network tasked with classifying handwritten digits may assign high confidence predictions to cat images. Anomalies are frequently encountered when deploying ML models in the real world. Well-calibrated predictive uncertainty estimates are indispensable for many machine learning applications, such as self-driving vehicles and medical diagnosis systems. Generalization to unseen and worst-case inputs is also essential for robustness to distributional shift. In order to have ML models reliably predict in open environment, we must deepen technical understanding in the emerging areas of: (1) learning algorithms that can detect changes in data distribution (e.g. out-of-distribution examples); (2) mechanisms to estimate and calibrate confidence produced by neural networks in typical and unforeseen scenarios; (3) methods to improve out-of-distribution generalization, including generalization to temporal, geographical, hardware, adversarial, and image-quality changes; (4) benchmark datasets and protocols for evaluating model performance under distribution shift; and (5) key applications of robust and uncertainty-aware deep learning (e.g., computer vision, robotics, self-driving vehicles, medical imaging) as well as broader machine learning tasks.
This workshop will bring together researchers and practitioners from the machine learning communities, and highlight recent work that contributes to addressing these challenges. Our agenda will feature contributed papers with invited speakers. Through the workshop we hope to help identify fundamentally important directions on robust and reliable deep learning, and foster future collaborations.
Fri 7:30 a.m. - 7:40 a.m.
|
Opening Remarks
(
Presentation
)
link »
SlidesLive Video » https://slideslive.com/38930571/opening-remarks-workshop-on-robustness-uncertainty-in-deep-learning |
Sharon Yixuan Li 🔗 |
Fri 7:40 a.m. - 8:10 a.m.
|
Keynote #1 Matthias Hein
(
Keynote
)
link »
SlidesLive Video » http://slideslive.com/38930572 |
Matthias Hein 🔗 |
Fri 8:10 a.m. - 8:15 a.m.
|
Spotlight Talk 1: Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder
(
Spotlight
)
link »
SlidesLive Video » https://slideslive.com/38930944 |
Zhisheng Xiao 🔗 |
Fri 8:15 a.m. - 8:20 a.m.
|
Spotlight Talk 2: A Closer Look at Accuracy vs. Robustness
(
Spotlight
)
link »
SlidesLive Video » https://slideslive.com/38930945 |
Yao-Yuan Yang 🔗 |
Fri 8:20 a.m. - 8:25 a.m.
|
Spotlight Talk 3: Depth Uncertainty in Neural Networks
(
Spotlight
)
link »
SlidesLive Video » https://slideslive.com/38930946 |
Javier Antorán · James Allingham 🔗 |
Fri 8:25 a.m. - 8:30 a.m.
|
Spotlight Talk 4: Few-shot Out-of-Distribution Detection
(
Spotlight
)
link »
SlidesLive Video » https://slideslive.com/38930947 |
Kuan-Chieh Wang 🔗 |
Fri 8:30 a.m. - 8:35 a.m.
|
Spotlight Talk 5: Detecting Failure Modes in Image Reconstructions with Interval Neural Network Uncertainty
(
Spotlight
)
link »
SlidesLive Video » https://slideslive.com/38930948 |
Luis Oala 🔗 |
Fri 8:35 a.m. - 8:40 a.m.
|
Spotlight Talk 6: On using Focal Loss for Neural Network Calibration
(
Spotlight
)
link »
SlidesLive Video » https://slideslive.com/38930949 |
Jishnu Mukhoti 🔗 |
Fri 8:40 a.m. - 8:45 a.m.
|
Spotlight Talk 7: AutoAttack: reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
(
Spotlight
)
link »
SlidesLive Video » https://slideslive.com/38930950 |
Francesco Croce 🔗 |
Fri 8:45 a.m. - 8:50 a.m.
|
Spotlight Talk 8: Calibrated Top-1 Uncertainty estimates for classification by score based models
(
Spotlight
)
link »
SlidesLive Video » https://slideslive.com/38930951 |
Adam Oberman 🔗 |
Fri 9:00 a.m. - 10:00 a.m.
|
Poster Session (click to see links)
(
Poster
)
1 Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks David Stutz, Matthias Hein and Bernt Schiele [ protected link dropped ] 2 Improving robustness against common corruptions by covariate shift adaptation Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel and Matthias Bethge https://meet.google.com/ojt-atoh-wup 3 A Unified View of Label Shift Estimation Saurabh Garg, Yifan Wu, Sivaraman Balakrishnan and Zachary Lipton https://cmu.zoom.us/j/94831507038?pwd=RVVyVy96YXNiaGtjMUxyZUJ5dVcvQT09 4 A Benchmark of Medical Out of Distribution Detection Tianshi Cao, David Yu-Tung Hui, Chin-Wei Huang and Joseph Paul Cohen https://meet.google.com/jxk-paui-gao 5 Neural Ensemble Search for Performant and Calibrated Predictions Sheheryar Zaidi, Arber Zela, Thomas Elsken, Chris Holmes, Frank Hutter and Yee Whye Teh https://meet.google.com/uxs-uuxr-uwo 6 Bayesian model averaging is suboptimal for generalization under model misspecification Andres Masegosa https://meet.google.com/mpf-chva-pgy 7 Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder Zhisheng Xiao, Qing Yan and Yali Amit https://uchicago.zoom.us/j/98748547880?pwd=Q3Y0dUVPUFphbGY4NmNJK2hwZndWUT09 Password: 370663 8 A Closer Look at Accuracy vs. Robustness Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Ruslan Salakhutdinov and Kamalika Chaudhuri https://ucsd.zoom.us/j/92778085557?pwd=STJVVWMza0RKSFZsOUVIS0h6bVl5QT09 9 Depth Uncertainty in Neural Networks Javier Antorán, James Urquhart Allingham and José Miguel Hernández-Lobato https://us02web.zoom.us/j/5419103161?pwd=eXdqWURLc3o4SktwQWZOU2pZNkliQT09 10 Few-shot Out-of-Distribution Detection Kuan-Chieh Wang, Paul Vicol, Eleni Triantafillou and Richard Zemel https://vectorinstitute.zoom.us/j/97199418667?pwd=NjBWUXhzZUwrOHFBMkxSZEd2MDh3Zz09 11 Detecting Failure Modes in Image Reconstructions with Interval Neural Network Uncertainty Luis Oala, Cosmas Heiß, Jan Macdonald, Maximilian März, Gitta Kutyniok and Wojciech Samek https://us02web.zoom.us/j/88251013741?pwd=TzlOUlJXeExzUVdSUDE5RXFkTytWZz09 12 On using Focal Loss for Neural Network Calibration Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip Torr and Puneet Dokania https://meet.google.com/ehz-rzxf-xeu 13 AutoAttack: reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks Francesco Croce and Matthias Hein [ protected link dropped ] 14 Calibrated Top-1 Uncertainty estimates for classification by score based models Adam Oberman, Chris Finlay, Alexander Iannantuono and Tiago Salvador https://mcgill.zoom.us/j/91308261627 15 Bayesian Deep Ensembles via the Neural Tangent Kernel Bobby He, Balaji Lakshminarayanan and Yee Whye Teh https://meet.google.com/exb-jkju-vbr 16 The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt and Justin Gilmer https://berkeley.zoom.us/j/4219480859?pwd=QkNIYmNCZjlobEUxQ2Q5TzR4Qm1QQT09 17 Measuring Robustness to Natural Distribution Shifts in Image Classification Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht and Ludwig Schmidt https://berkeley.zoom.us/j/97514838599?pwd=SXdiOSthMzNUV01QYU5lVElFTTM2dz09 18 Redundant features can hurt robustness to distribution shift Guillermo Ortiz-Jimenez, Apostolos Modas, Seyed-Mohsen Moosavi-Dezfooli and Pascal Frossard https://epfl.zoom.us/j/99757698661?pwd=NVc5OUVHenFYc2pXVlFYbk12U0F4dz09 548657 19 Scalable Training with Information Bottleneck Objectives Andreas Kirsch, Clare Lyle and Yarin Gal https://meet.google.com/mzn-pjuh-kia 20 How Does Early Stopping Help Generalization Against Label Noise? Hwanjun Song Hwanjun, Minseok Kim, Dongmin Park and Jae-Gil Lee [ protected link dropped ] 21 Soft Labeling Affects Out-of-Distribution Detection of Deep Neural Networks Doyup Lee and Yeongjae Cheon [ protected link dropped ] 0deFHw 22 Learning Robust Representations with Score Invariant Learning Daksh Idnani and Jonathan Kao https://ucla.zoom.us/j/98883773408?pwd=Y3M2WmxNTW1HOS9hMlBZYmJJSHZ0dz09 23 PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction Sangdon Park, Osbert Bastani, Nikolai Matni and Insup Lee https://us04web.zoom.us/j/76965715700?pwd=U2tNWVBMd0RKMHcrNGlTa0VMQkY3dz09 24 Learned Uncertainty-Aware (LUNA) Bases for Bayesian Regression using Multi-Headed Auxiliary Networks Sujay Thakur, Cooper Lorsung, Yaniv Yacoby, Finale Doshi-Velez and Weiwei Pan https://harvard.zoom.us/j/96201359957?pwd=N3Q3RVJJZ0R0TmRuejYxTDRDMjAxUT09 25 Domain Generalization using Causal Matching Divyat Mahajan, Shruti Tople and Amit Sharma https://us02web.zoom.us/j/87167858273?pwd=V09XWTdjMzVSTHplcnJ6dCtpUWphUT09 26 Hydra: Preserving Ensemble Diversity for Model Distillation Linh Tran, Bastiaan S. Veeling, Kevin Roth, Jakub Swiatkowski, Joshua V. Dillon, Stephan Mandt, Jasper Snoek, Tim Salimans, Sebastian Nowozin and Rodolphe Jenatton meet.google.com/vqj-zckz-hbr 27 Predicting with High Correlation Features Devansh Arpit, Caiming Xiong and Richard Socher https://us04web.zoom.us/j/3643443782 28 Consistency Regularization for Certified Robustness of Smoothed Classifiers Jongheon Jeong and Jinwoo Shin https://us02web.zoom.us/j/85431063603?pwd=MGJ4YzR4Q2FXSWM4c3c3NURiR3NqZz09 29 DIBS: Diversity inducing Information Bottleneck in Model Ensembles Samarth Sinha, Homanga Bharadhwaj, Anirudh Goyal, Hugo Larochelle, Animesh Garg and Florian Shkurti https://utoronto.zoom.us/j/6995996022?pwd=UUIxNUNnQThLVmhLV0h5RDVJRG52dz09 30 On the relationship between class selectivity, dimensionality, and robustness Matthew Leavitt and Ari Morcos https://fb.zoom.us/j/98555977785 949102 31 Robust Variational Autoencoder for Tabular Data with β Divergence Haleh Aydore, Sergul Aydore, Richard Leahy and Anand Joshi https://usc.zoom.us/j/92860013817?pwd=aG5YekNUcFBIalV4TVNJYkhyb1UzZz09 906642 32 Uncertainty in Structured Prediction Andrey Malinin and Mark Gales https://yandex.zoom.us/j/98605165946?pwd=RkJ4RTVmYmt6eU1CcWU5RGgvNEMzdz09 33 Understanding and Improving Fast Adversarial Training Maksym Andriushchenko and Nicolas Flammarion https://epfl.zoom.us/j/98065886762?pwd=dTVFMUdFZllIZFVEMllDbkdxL2QvZz09 34 Predictive Complexity Priors Eric Nalisnick, Jonathan Gordon and Jose Miguel Hernandez Lobato https://meet.google.com/jyk-qhvg-kek 35 Provable Worst Case Guarantees for the Detection of Out-of-Distribution Data Julian Bitterwolf, Alexander Meinke and Matthias Hein [ protected link dropped ] 36 Ensemble Distribution Distillation via Regression Prior Networks Andrey Malinin, Sergey Chervontsev, Ivan Provilkov and Mark Gales meet.google.com/zbn-vwtv-eda 37 Generalizing to unseen domains via distribution matching Isabela Albuquerque, Joao Monteiro, Mohammad Darvishi, Tiago Falk and Ioannis Mitliagkas [ protected link dropped ] 2U0Qsc 38 GAN-mixup: Augmenting Across Data Manifolds for Improved Robustness Jy-Yong Sohn, Kangwook Lee, Jaekyun Moon and Dimitris Papailiopoulos https://us02web.zoom.us/j/84163398388?pwd=MGFXWm9VZGhDZ01mMXFCMkNJckdRdz09 240878 39 Improving out-of-distribution generalization via multi-task self-supervised pretraining Isabela Albuquerque, Nikhil Naik, Junnan Li, Nitish Shirish Keskar and Richard Socher [ protected link dropped ] 6byCUy 40 Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks Shreyas Padhy, Zachary Nado, Jie Ren, Jeremiah Liu, Jasper Snoek and Balaji Lakshminarayanan https://meet.google.com/cym-bnec-mbb 41 Tilted Empirical Risk Minimization Tian Li, Ahmad Beirami, Maziar Sanjabi and Virginia Smith https://cmu.zoom.us/j/96622035970?pwd=M21WMTllV0tuWkpMaUlHMDlBWmpFdz09 42 Towards Robust Classification with Deep Generative Forests Alvaro Henrique Chaim Correia, Robert Peharz and Cassio de Campos [ protected link dropped ] 43 Riemannian Continuous Normalizing Flows Emile Mathieu https://meet.google.com/vts-axeh-doo 44 Improving Calibration of BatchEnsemble with Data Augmentation Yeming Wen, Ghassen Jerfel, Rafael Muller, Michael Dusenberry, Jasper Snoek, Balaji Lakshminarayanan and Dustin Tran https://us04web.zoom.us/j/79191629966?pwd=eVNEZ2NRVGY4NEc0Uk1DK2t1Ykh6QT09. 45 Environment Inference for Invariant Learning Elliot Creager, Jörn-Henrik Jacobsen and Richard Zemel https://vectorinstitute.zoom.us/j/98611353513?pwd=Mk5xa2VqLzVBV0JzdjZINXNlWXJMdz09 470387 46 Nonlinear Gradient Estimation for Query Efficient Blackbox Attack Huichen Li, Linyi Li, Xiaojun Xu, Xiaolu Zhang, Shuang Yang and Bo Li https://illinois.zoom.us/j/92637811021?pwd=WldhQXRVaENIWXFNQjdrdXhmcG1UQT09 NLBA 47 ImageNet performance correlates with pose estimation robustness and generalization on out-of-domain data Alexander Mathis, Thomas Biasi, Mert Yüksegönül, Byron Rogers, Matthias Bethge and Mackenzie Mathis https://harvard.zoom.us/j/96654497889?pwd=Z2NrL05lQlozZndsQ25jV0JQdkRtZz09 48 CRUDE: Calibrating Regression Uncertainty Distributions Empirically Eric Zelikman, Christopher Healy, Sharon Zhou and Anand Avati https://stanford.zoom.us/j/92845303885?pwd=Kzd6d2tjMGdXL0VkK01JM09jYkVqUT09 519357 49 Positive-Unlabeled Learning with Arbitrarily Non-Representative Labeled Data Zayd Hammoudeh and Daniel Lowd https://uoregon.zoom.us/j/99760928750?pwd=dmVCUlI3WUlxNldDRGVqQmQrSDVFQT09 359623 50 Probabilistic Robustness Estimates for Deep Neural Networks Nicolas Couellan [ protected link dropped ] 51 Estimating Risk and Uncertainty in Deep Reinforcement Learning William Clements, Bastien Van Delft, Benoît-Marie Robaglia, Reda Bahi Slaoui and Sébastien Toth https://us02web.zoom.us/j/84543888187?pwd=cVdIS1ZjQUVzaTNkR0lRT2lTVzNxZz09 52 An Empirical Study of Invariant Risk Minimization Yo Joong Choe, Jiyeon Ham and Kyubyong Park https://cmu.zoom.us/j/97184088310?pwd=MGJ0bEYzQXg3RVhHSkxTTWY2RnBHdz09 53 Information-Bottleneck under Mean Field Initialization Vinayak Abrol and Jared Tanner [ protected link dropped ] 54 Adaptive Risk Minimization: A Meta-Learning Approach for Tackling Group Shift Marvin Zhang, Henrik Marklund, Abhishek Gupta, Sergey Levine and Chelsea Finn https://berkeley.zoom.us/j/96584472302?pwd=TXlPWHFzeHhVTzROYnB4KzJVSVJhUT09 55 Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift Zachary Nado, Shreyas Padhy, D. Sculley, Alexander D’amour, Balaji Lakshminarayanan and Jasper Snoek https://meet.google.com/zxr-qijn-xkx 56 Failures of Variational Autoencoders and their Effects on Downstream Tasks Yaniv Yacoby, Weiwei Pan and Finale Doshi-Velez https://harvard.zoom.us/j/98541353638?pwd=QUovTjVKYm4rM1NSamw1VUJ3eVI2Zz09 57 Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness Jeremiah Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax-Weiss and Balaji Lakshminarayanan [ protected link dropped ] 58 Bayesian BERT for Trustful Hate Speech Detection Kristian Miok, Blaz Skrlj, Daniela Zaharie and Marko Robnik-Sikonja [ protected link dropped ] 12341234 59 On the Role of Dataset Quality and Heterogeneity in Model Confidence Yuan Zhao, Jiasi Chen and Samet Oymak https://us04web.zoom.us/j/76373416589?pwd=L09lNlBDUXcvNHZ6MjV3TTJUQzUxQT0 60 Certified Adversarial Robustness via Randomized Smoothing: a Case Study for Laplace Noises Jiaye Teng, Guanghe Lee and Yang Yuan [ protected link dropped ] password 3QAa4p 61 QUEST for MEDISYN: Quasi-norm based Uncertainty ESTimation for MEDical Image SYNthesis Uddeshya Upadhyay, Viswanath P. Sudarshan and Suyash P. Awate https://monash.zoom.us/j/4586078207?pwd=OGlzeHh0bVovM3lkNCttRW40SFNMZz09 quest 62 On uncertainty estimation in active learning for image segmentation Bo Li and Tommy Sonne Alstrøm https://dtudk.zoom.us/j/65926847472?pwd=NGU0cXVlMy9oeGQ3dmp5WlFCcWgvUT09 63 Bayesian Autoencoders: Analysing and Fixing the Bernoulli likelihood for Out-of-Distribution Detection Bang Xiang Yong, Tim Pearce and Alexandra Brintrup meet.google.com/uxr-hzin-coj 64 A Comparison of Bayesian Deep Learning for Out of Distribution Detection and Uncertainty Estimation John Mitros, Arjun Pakrashi and Brian Mac Namee https://meet.google.com/puu-yysf-owt?hs=122&authuser=0 65 Practical Bayesian Neural Networks via Adaptive Subgradient Optimization Methods Samuel Kessler, Arnold Salas, Vincent Tan Weng Choon, Stefan Zohren and Stephen Roberts https://us02web.zoom.us/j/81195534206 66 Bayesian Few-Shot Classification with One-vs-Each Pólya-Gamma Augmented Gaussian Processes Jake Snell and Richard Zemel https://vectorinstitute.zoom.us/j/99320884951?pwd=aEdhbE8wUWNIS1ZuZkc4ZHFKVGNWUT09 67 Characteristics of Monte Carlo Dropout in Wide Neural Networks Joachim Sicking, Maram Akila, Tim Wirtz, Sebastian Houben and Asja Fischer https://us02web.zoom.us/j/88053290381?pwd=T2FaaXk1aUhWcS9iSFFhTW9WY1NTdz09 68 On Power Laws in Deep Ensembles Ekaterina Lobacheva, Nadezhda Chirkova, Maxim Kodryan and Dmitry Vetrov [ protected link dropped ] 69 Outlier Detection through Null Space Analysis of Neural Networks Matthew Cook, Alina Zare and Paul Gader https://ufl.zoom.us/j/94556341852?pwd=MXRoVUw5M3BUMmFYcjlGOTBVdDhIQT09 70 In a forward direction: Analyzing distribution shifts in machine translation test sets over time Thomas Liao, Benjamin Recht and Ludwig Schmidt https://berkeley.zoom.us/j/98684240218?pwd=elZ3RVVOQllkdDZGTU1PZTY0dGZPUT09 71 Characterizing Adversarial Transferability via Gradient Orthogonality and Smoothness Zhuolin Yang, Linyi Li, Xiaojun Xu, Kaizhao Liang, Shiliang Zuo, Qian Chen, Benjamin Rubinstein, Ce Zhang and Bo Li https://illinois.zoom.us/j/95161238471?pwd=NHRrZ1c0ZlAwMVp6U2Vwd3U2OXk1dz09 926964 72 Untapped Potential of Data Augmentation: A Domain Generalization Viewpoint Vihari Piratla and Shiv Shankar https://meet.google.com/xya-jmcm-htd 73 Classifying Perturbation Types for Adversarial Robustness Against Multiple Threat Models Pratyush Maini, Xinyun Chen, Bo Li and Dawn Song https://berkeley.zoom.us/j/96355581753?pwd=djFrc0hMVmQ1RU5rMzNsbCtjbXVQdz09 74 A Critical Evaluation of Open-World Machine Learning Liwei Song, Vikash Sehwag, Arjun Nitin Bhagoji and Prateek Mittal https://princeton.zoom.us/j/6441846264?pwd=RWgzYjA2aWJVbnJvaVcvc2FQRm1HUT09 75 You won't believe you can learn CIFAR-10 with this: Another take on Information Bottleneck Objectives Andreas Kirsch, Clare Lyle and Yarin Gal https://meet.google.com/tes-egvb-nso 76 Principled Uncertainty Estimation for High Dimensional Data Pascal Notin, José Miguel Hernández-Lobato and Yarin Gal https://meet.google.com/xte-ywjb-paj 77 Rethink Autoencoders: Robust Manifold Learning Taihui Li, Rishabh Mehta, Zecheng Qian and Ju Sun https://umn.zoom.us/j/92216511031?pwd=UjN1K2IvRFVvaDlpVUlXZWFqMURZUT09 7zcn93 78 Continuous-Depth Bayesian Neural Networks Winnie Xu, Ricky T.Q. Chen and David Duvenaud https://utoronto.zoom.us/j/7553398989 755996 79 Robust Temporal Point Event Localization through Smoothing and Counting Julien Schroeter, Kirill Sidorov and David Marshall https://cardiff.zoom.us/j/96449530903?pwd=RjBtNDFmY0ZlT1EyMFJRZjdNdDh0UT09 80 Chi-square Information for Invariant Learning Prasanna Sattigeri, Soumya Ghosh and Samuel Hoffman https://us04web.zoom.us/j/77772482601?pwd=MXdDTDRUWmFJbktYcWVkT3NRRWUxdz09 81 Robust Out-of-distribution Detection via Informative Outlier Mining Jiefeng Chen, Sharon Li, Xi Wu, Yingyu Liang and Somesh Jha https://stanford.zoom.us/j/99582468298?pwd=V1VINTI1U1JJalRBaktTQnk4c0VvQT09 82 An Empirical Analysis of the Impact of Data Augmentation on Distillation Deepan Das, Haley Massa, Abhimanyu Kulkarni and Theodoros Rekatsinas https://meet.google.com/yzd-nbjv-qyx 83 It Is Likely That Your Loss Should be a Likelihood Mark Hamilton, Evan Shelhamer and William Freeman https://mit.zoom.us/j/98388816271?pwd=cTZscGVtNzhzTXUzUGxTcUtpVkh3dz09 udl 84 Self-Adaptive Training: beyond Empirical Risk Minimization Huang, Chao Zhang and Hongyang Zhang https://meet.google.com/cek-gsff-ebv 85 BaCOUn: Bayesian Classifers with Out-of-Distribution Uncertainty Théo Guenais, Dimitris Vamvourellis, Yaniv Yacoby, Finale Doshi-Velez and Weiwei Pan https://us02web.zoom.us/j/84495842174?pwd=czNvaDkrMmZGRVgvdjlEczJxWUJwUT09 86 Bayesian active learning for production, a systematic study and a reusable library Parmida Atighehchian, Frédéric Branchaud-Charron and Alexandre Lacoste https://elementai.zoom.us/j/92111241672?pwd=NVg2V2Myd1lmKzFaWXlCa1dpNHZMUT09 87 Certainty as Supervision for Test-Time Adaptation Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen and Trevor Darrell https://berkeley.zoom.us/j/92105695905?pwd=cEk0T0U5Y0RDb2ZFSGZ2b09HTStIdz09 88 Robust Deep Reinforcement Learning through Adversarial Loss Tuomas Oikarinen, Tsui-Wei Weng and Luca Daniel https://mit.zoom.us/j/92724256949?pwd=K3JRdHhhVG1xV2tRbE81WXJQUGZsdz09 89 Cold Posteriors and Aleatoric Uncertainty Ben Adlam, Sam Smith and Jasper Snoek meet.google.com/ncb-ffxx-qgc 90 Ensemble Mean vs. Ensemble Variance: Which is a Better Uncertainty Metric for Incipient Disease Detection? Baihong Jin, Yingshui Tan, Xiangyu Yue, Yuxin Chen and Alberto Sangiovanni-Vincentelli https://berkeley.zoom.us/j/95691253740?pwd=SDFlR2hXYkRBQWFJZ3cydXlwRnEyZz09 91 Simplicity Bias and the Robustness of Neural Networks Harshay Shah, Kaustav Tamuly, Aditi Raghunathan, Prateek Jain and Praneeth Netrapalli https://meet.google.com/bdq-nckv-tyq 92 Exact posterior distributions of wide Bayesian neural networks Jiri Hron, Yasaman Bahri, Roman Novak, Jeffrey Pennington and Jascha Sohl-Dickstein https://meet.google.com/pvb-brkx-png 93 A Simulation-based Framework for Characterizing Predictive Distributions for Deep Learning Jessica Ai, Beliz Gokkaya, Ilknur Kaynar Kabul, Erik Meijer, Audrey Flower, Ehsan Emamjomeh-Zadeh, Hannah Li, Li Chen, Neamah Hussein, Ousmane Dia and Sevi Baltaoglu https://us04web.zoom.us/j/72581014579?pwd=MFlJeXRKZTBJdWw4eDNoeTJEcGZSUT09 94 Structured Weight Priors for Convolutional Neural Networks Tim Pearce, Andrew Y.K. Foong and Alexandra Brintrup meet.google.com/rzg-vdpa-xir 95 Learning Generative Models from Classifier Uncertainties Siddharth Narayanaswamy and Brooks Paige https://meet.google.com/gjz-innd-iyt 96 DQI: A Guide to Benchmark Evaluation Swaroop Mishra, Anjana Arunkumar, Bhavdeep Sachdeva, Chris Bryan and Chitta Baral https://asu.zoom.us/j/95489346758 97 Can Your AI Differentiate Cats from Covid-19? Sample Efficient Uncertainty Estimation for Deep Learning Safety Ankur Mallick, Chaitanya Dwivedi, Bhavya Kailkhura, Gauri Joshi and T. Yong-Jin Han https://cmu.zoom.us/j/95091389695?pwd=K2RXbzZVSitzRVZadHB4eHZaZmlDZz09 98 Transferable Adversarial Examples for Atari 2600 Games Damian Stachura and Michał Zając https://meet.google.com/jwf-nqyd-fty 99 Robustness to Distribution Shifts using Multiple Environments Anders Andreassen, Rebecca Roelofs and Behnam Neyshabur [ protected link dropped ] 100 Our Evaluation Metric Needs an Update to Encourage Generalization Swaroop Mishra, Anjana Arunkumar and Chris Bryan https://asu.zoom.us/j/92127915576 101 Harder or Different? A Closer Look at Distribution Shift in Dataset Reproduction Shangyun Lu, Bradley Nott, Aaron Olson, Alberto Todeschini, Puya Vahabi, Yair Carmon and Ludwig Schmidt https://stanford.zoom.us/j/98068202076?pwd=LytZdjg5bTR5eldVcFFHQVpSaHRzZz09 102 Empirical Scoring Rule Decomposition in Deep Learning Tony Duan https://teams.microsoft.com/l/meetup-join/19%3ameeting_NTBkNzE2OTYtODQwOS00MjI0LTgyNjQtY2Q3YWRhN2M1ZWNj%40thread.v2/0?context=%7b%22Tid%22%3a%2272f988bf-86f1-41af-91ab-2d7cd011db47%22%2c%22Oid%22%3a%2233328e52-be2d-485a-a0c5-7a8d97aaa4e2%22%7d 103 Failure Prediction by Confidence Estimation of Uncertainty-Aware Dirichlet Networks Theodoros Tsiligkaridis https://mit.zoom.us/j/95278652551?pwd=WHNwNEsxUUVOd0FSaUVHbkJaNHp5QT09 140350 104 RayS: A Ray Searching Method for Hard-label Adversarial Attack Jinghui Chen and Quanquan Gu https://ucla.zoom.us/j/92290062678?pwd=MThrUVVDSnlEU2FNWG10Yk1CcnRlZz09 987689 105 Joint Energy-Based Models for Semi-Supervised Classification Stephen Zhao, Joern-Henrik Jacobsen and Will Grathwohl https://us02web.zoom.us/j/88327646584?pwd=ZkxBcDNRMU9WVUt0ODJ6SDhoYk5zZz09 231335 106 Single Shot MC Dropout Approximation Kai Brach, Beate Sick and Oliver Duerr https://us02web.zoom.us/j/89892013425?pwd=VDZQN2JLSy9hTVg3VGpqOFhBdVJiZz09 1yqhv2 107 Reliable Uncertainties for Bayesian Neural Networks using Alpha-divergences Hector Javier Hortua, Luigi Malago and Riccardo Volpi [ protected link dropped ] 8a3QWf 108 Evaluating Uncertainty Estimation Methods on 3D Semantic Segmentation of Point Clouds Swaroop Bhandary, Nico Hochgeschwender, Paul Plöger and Matias Valdenegro-Toro [ protected link dropped ] 109 Improving predictions of Bayesian neural networks via local linearization Alexander Immer, Maciej Korzepa and Matthias Bauer https://ethz.zoom.us/j/98684846685?pwd=aVJFS3VNdSsyMGVzOFQvbW5qSisrZz09 110 URSABench: Comprehensive Benchmarking of Approximate Bayesian Inference Methods for Deep Neural Networks Meet Vadera, Adam Cobb, Brian Jalaian and Benjamin Marlin https://umass-amherst.zoom.us/j/99916090662?pwd=N1E1Mm9LcEQ4bmhjNWlWdTlyU3BGZz09 111 Richness of Training Data Does Not Suffice: Robustness of Neural Networks Requires Richness of Hidden-Layer Activations Kamil Nar and Shankar Sastry https://berkeley.zoom.us/j/98295724751?pwd=cW5QQ2EzL2RaUW15OWs3T3JrTjk2Zz09 112 Robust Classification under Class-Dependent Domain Shift Tigran Galstyan, Hrant Khachatrian, Greg Ver Steeg and Aram Galstyan https://us02web.zoom.us/j/82576216178?pwd=RzVGQlZUNE95S1NrRktaRk5WTXMrdz09 736349 113 You Need Only Uncertain Answers: Data Efficient Multilingual Question Answering Zhihao Lyu, Danier Duolikun, Bowei Dai, Yuan Yao, Pasquale Minervini, Tim Z. Xiao and Yarin Gal https://us02web.zoom.us/j/88327646584?pwd=ZkxBcDNRMU9WVUt0ODJ6SDhoYk5zZz09 231335 114 Diverse Ensembles Improve Calibration Asa Cooper Stickland and Iain Murray [ protected link dropped ] 115 Exploring the Uncertainty Properties of Neural Networks’ Implicit Priors in the Infinite-Width Limit Ben Adlam, Jaehoon Lee, Lechao Xiao, Jeffrey Pennington and Jasper Snoek meet.google.com/agb-migr-qdt 116 Provable Robust Learning Based on Transformation-Specific Smoothing Linyi Li, Maurice Weber, Xiaojun Xu, Luka Rimanic, Shuang Yang, Tao Xie, Ce Zhang and Bo Li https://ethz.zoom.us/j/98321715637?pwd=Zk1hS0J1NUlteWYxUmtUYVlXb2hOdz09 117 Simple and Effective VAE Training with Calibrated Decoders Oleh Rybkin, Kostas Daniilidis and Sergey Levine meet.google.com/hvr-sdbx-syc 118 Amortized Conditional Normalized Maximum Likelihood Aurick Zhou and Sergey Levine https://berkeley.zoom.us/j/4913207255?pwd=VGN5bXRJQi9HRXJEbFp6SjZ3a29idz09 119 Neural Networks with Recurrent Generative Feedback Yujia Huang, James Gornet, Sihui Dai, Zhiding Yu, Tan Nguyen, Doris Y. Tsao and Anima Anandkumar https://us02web.zoom.us/j/6830799787?pwd=NGU3VFRvWTdhbTNiK1YzVHgzMEs2QT09 9appDa 120 Robustness of Latent Representations of Variational Autoencoders Andrea Karlova https://meet.google.com/hmk-kory-kba 121 Learning approximate invariance requires far fewer data Jean Michel Amath Sarr, Alassane Bah and Christophe Cambier https://us04web.zoom.us/j/75476163865?pwd=Z3RES1dMWFVWZkE4YjM2RzRLWUJ6Zz09 122 MCU-Net: A framework towards uncertainty representations for decision support system patient referrals in healthcare contexts Nabeel Seedat [ protected link dropped ] 123 Uncertainty-sensitive Learning and Planning with Ensembles Piotr Milos, Lukasz Kucinski, Konrad Czechowski, Piotr Kozakowski and Maciej Klimek https://us02web.zoom.us/j/89300913614?pwd=cVJjdFRpaDFZRVRMQXBsSzZzOGVSQT09 124 Uncertainty in Multi-Interaction Trajectory Reconstruction Vasileios Karavias, Ben Day and Pietro Lio https://meet.google.com/uqx-qudh-ish 125 Unsupervised Domain Adaptation in the Absence of Source Data Roshni Sahoo, Divya Shanmugam and John Guttag https://mit.zoom.us/j/94674840454?pwd=L2NMaHBsaTRYZnE4TjZydmJERCtqUT09 126 A benchmark study on reliable molecular supervised learning via Bayesian learning Doyeong Hwang, Grace Lee, Hanseok Jo, Seyoul Yoon and Seongok Ryu https://meet.google.com/hiy-bbdm-qxo 127 On the Power of Oblivious Poisoning Attacks Samuel Deng, Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mahmoody and Abhradeep Thakurta https://virginia.zoom.us/j/97053762096 128 Training Deep Neural Networks with Class-level Semantics for Explicable Classification Alberto Olmo, Sailik Sengupta and Subbarao Kambhampati https://asu.zoom.us/j/93814557218?pwd=K1hjc3NqWXlLQm1pN0tHTk1rdUNJZz09 129 End-to-end Robustness for Sensing-Reasoning Machine Learning Pipelines Zhuolin Yang, Zhikuan Zhao, Hengzhi Pei, Boxin Wang, Bojan Karlas, Ji Liu, Heng Guo, Bo Li and Ce Zhang https://illinois.zoom.us/j/2530610336?pwd=WXVZYlkreDYyUjRNdFh0Qk1IMURjUT09 password 633681 130 Deep Robust Classification under Domain Shift with Conservative Uncertainty Estimation Haoxuan Wang, Anqi Liu, Yisong Yue and Anima Anandkumar https://caltech.zoom.us/j/92474618222?pwd=WS91UjhVY1p4WThHQjRRYU5LMnpJQT09 131 On Separability of Self-Supervised Representations Vikash Sehwag, Mung Chiang and Prateek Mittal https://princeton.zoom.us/j/3871410688?pwd=RldBOUNwMnhFU3d4ZGpRSVR2YWJSdz09 132 Dropout Strikes Back: Improved Uncertainty Estimation via Diversity Sampling Evgenii Tsymbalov, Kirill Fedyanin and Maxim Panov [ protected link dropped ] 133 Predictive Uncertainty for Probabilistic Novelty Detection in Text Classification Jordy Van Landeghem, Matthew Blaschko, Bertrand Anckaert and Marie-Francine Moens https://us02web.zoom.us/j/86332883100?pwd=TVd3UytVTkF1R1V1RlNpdTZqbWlaZz09 134 Maximizing the Representation Gap between In-domain & OOD examples Jay Nandy, Wynne Hsu and Mong Li Lee https://nus-sg.zoom.us/j/99627439405?pwd=WFlkNzhndlo3TTU5YytyenduN0NWQT09 135 Regional Image Perturbation Reduces Lp Norms of Adversarial Examples While Maintaining Model-to-model Transferability Utku Ozbulak, Jonathan Peck, Wesley De Neve, Bart Goossens, Yvan Saeys and Arnout Van Messem [ protected link dropped ] 135 136 Lookahead Adversarial Semantic Segmentation Hadi Jamali-Rad, Attila Szabo and Matteo Presutto meet.google.com/jgc-nfyp-nsn 137 Neural Network Calibration for Medical Imaging Classification Using DCA Regularization Gongbo Liang, Yu Zhang and Nathan Jacobs https://uky.zoom.us/j/93398011263?pwd=RUlBYzU3N1N4ekdzWEdZZ0czSnJpQT09 138 Deep k-NN Defense Against Clean-Label Data Poisoning Attacks Neehar Peri, Neal Gupta, Ronny Huang, Chen Zhu, Liam Fowl, Soheil Feizi, Tom Goldstein and John Dickerson https://umd.zoom.us/j/93698080878?pwd=S2VjZjAzZHB3RXlSUWtiRWh6K1N0dz09 139 Amortised Variational Inference for Hierarchical Mixture Models Javier Antoran, Jiayu Yao, Weiwei Pan, Finale Doshi-Velez and Jose Miguel Hernandez-Lobato https://harvard.zoom.us/j/95984006384?pwd=N09BSlRGWWR5T1czNE5vQjBGbm9jdz09 140 Benchmarking Search Methods for Generating NLP Adversarial Examples Jin Yong Yoo, John X. Morris, Eli Lifland and Yanjun Qi https://virginia.zoom.us/j/97965655398?pwd=aWd2ZW1EK2l1Y2wzL3ZMVGF0QW1rQT09 |
🔗 |
Fri 10:00 a.m. - 10:30 a.m.
|
Coffee Break
|
🔗 |
Fri 10:30 a.m. - 11:00 a.m.
|
Keynote #2 Finale Doshi-Velez
(
Keynote
)
link »
SlidesLive Video » http://slideslive.com/38930573 |
Finale Doshi-Velez 🔗 |
Fri 11:00 a.m. - 11:30 a.m.
|
Keynote #3 Percy Liang
(
Keynote
)
link »
SlidesLive Video » http://slideslive.com/38930574 |
Percy Liang 🔗 |
Fri 11:30 a.m. - 12:30 p.m.
|
Panel Discussion
(
Panel
)
link »
https://us02web.zoom.us/s/85455699146?pwd=REw5QkhTNk5MV3cxUVp5VnNzRWxhZz09 |
🔗 |
Fri 12:30 p.m. - 1:30 p.m.
|
Lunch Break
|
🔗 |
Fri 1:30 p.m. - 2:00 p.m.
|
Keynote #4 Raquel Urtasun
(
Keynote
)
link »
SlidesLive Video » http://slideslive.com/38930575 |
Raquel Urtasun 🔗 |
Fri 2:00 p.m. - 2:10 p.m.
|
Contributed Talk 1: Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks
(
Presentation
)
link »
SlidesLive Video » http://slideslive.com/38930576 |
David Stutz 🔗 |
Fri 2:10 p.m. - 2:20 p.m.
|
Contributed Talk 2: Improving robustness against common corruptions by covariate shift adaptation
(
Presentation
)
link »
SlidesLive Video » http://slideslive.com/38930577 |
Steffen Schneider 🔗 |
Fri 2:20 p.m. - 2:30 p.m.
|
Contributed Talk 3: A Unified View of Label Shift Estimation
(
Presentation
)
link »
SlidesLive Video » https://slideslive.com/38930578 |
Saurabh Garg 🔗 |
Fri 2:30 p.m. - 3:00 p.m.
|
Keynote #5 Justin Gilmer
(
Keynote
)
link »
SlidesLive Video » https://slideslive.com/38930579 |
Justin Gilmer 🔗 |
Fri 3:00 p.m. - 3:30 p.m.
|
Coffee Break
|
🔗 |
Fri 3:30 p.m. - 3:40 p.m.
|
Contributed Talk 4: A Benchmark of Medical Out of Distribution Detection
(
Presentation
)
link »
SlidesLive Video » https://slideslive.com/38930580 |
Joseph Paul Cohen 🔗 |
Fri 3:40 p.m. - 3:50 p.m.
|
Contributed Talk 5: Neural Ensemble Search for Performant and Calibrated Predictions
(
Presentation
)
link »
SlidesLive Video » https://slideslive.com/38930582 |
Sheheryar Zaidi 🔗 |
Fri 3:50 p.m. - 4:00 p.m.
|
Contributed Talk 6: Bayesian model averaging is suboptimal for generalization under model misspecification
(
Presentation
)
link »
SlidesLive Video » https://slideslive.com/38930581 |
Andres Arrendondo 🔗 |
Author Information
Sharon Yixuan Li (Stanford University)
Sharon Y. Li is currently a postdoc researcher in the Computer Science department at Stanford, working with Chris Ré. She will be joining the Computer Sciences Department at University of Wisconsin Madison as an assistant professor, starting in Fall 2020. Previously, she completed her PhD from Cornell University in 2017, where she was advised by John E. Hopcroft. Her thesis committee members are Kilian Q. Weinberger and Thorsten Joachims. She has spent time at Google AI twice as an intern, and Facebook AI as a Research Scientist. She was named Forbes 30 Under 30 in Science in 2020. Her principal research interests are in the algorithmic foundations of deep learning and its applications. Her time in both academia and industry has shaped my view and approach in research. She is particularly excited about developing open-world machine learning methods that can reduce human supervision during training, and enhance reliability during deployment.
Balaji Lakshminarayanan (Google Brain)
Dan Hendrycks (UC Berkeley)
Thomas Dietterich (Oregon State University)
Jasper Snoek (Google Brain)
More from the Same Authors
-
2021 : A simple fix to Mahalanobis distance for improving near-OOD detection »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Wide Mean-Field Variational Bayesian Neural Networks Ignore the Data »
Stanislav Fort · Jasper Snoek -
2021 : Precise characterization of the prior predictive distribution of deep ReLU networks »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Disentangling the Roles of Curation, Data-Augmentation and the Prior in the Cold Posterior Effect »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Exploring the Limits of Out-of-Distribution Detection »
Jasper Snoek -
2021 : Repulsive Deep Ensembles are Bayesian »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Calibrated Out-of-Distribution Detection with Conformal P-values »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Are Bayesian neural networks intrinsically good at out-of-distribution detection? »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Provably Robust Detection of Out-of-distribution Data (almost) for free »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Out-of-Distribution Dynamics Detection: RL-Relevant Benchmarks and Results »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Rethinking Assumptions in Deep Anomaly Detection »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Multiple Moment Matching Inference: A Flexible Approximate Inference Algorithm »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : PAC Prediction Sets Under Covariate Shift »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Correct-N-Contrast: a Contrastive Approach for Improving Robustness to Spurious Correlations »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Do We Really Need to Learn Representations from In-domain Data for Outlier Detection? »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : DATE: Detecting Anomalies in Text via Self-Supervision of Transformers »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantification »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Implicit Ensemble Training for Efficient and Robust Multiagent Reinforcement Learning »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Failures of Uncertainty Estimation on Out-Of-Distribution Samples: Experimental Results from Medical Applications Lead to Theoretical Insights »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : On Out-of-distribution Detection with Energy-Based Models »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Deterministic Neural Networks with Inductive Biases Capture Epistemic and Aleatoric Uncertainty »
Andreas Kirsch · Balaji Lakshminarayanan · Jasper Snoek -
2021 : Transfer and Marginalize: Explaining Away Label Noise with Privileged Information »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Meta-Calibration: Meta-Learning of Model Calibration Using Differentiable Expected Calibration Error »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Inferring Black Hole Properties from Astronomical Multivariate Time Series with Bayesian Attentive Neural Processes »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Towards improving robustness of compressed CNNs »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : SAND-mask: An Enhanced Gradient Masking Strategy for Invariant Prediction in Domain Generalization »
Soroosh Shahtalebi · Jasper Snoek · Balaji Lakshminarayanan -
2021 : Efficient Gaussian Neural Processes for Regression »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Simple, Attack-Agnostic Defense Against Targeted Training Set Attacks Using Cosine Similarity »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Safety & Exploration: A Comparative Study of Uses of Uncertainty in Reinforcement Learning »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Rethinking Function-Space Variational Inference in Bayesian Neural Networks »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Understanding the Under-Coverage Bias in Uncertainty Estimation »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : BETH Dataset: Real Cybersecurity Data for Anomaly Detection Research »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Mean Embeddings with Test-Time Data Augmentation for Ensembling of Representations »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Deep Ensemble Uncertainty Fails as Network Width Increases: Why, and How to Fix It »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Exact and Efficient Adversarial Robustness with Decomposable Neural Networks »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Consistency Regularization for Training Confidence-Calibrated Classifiers »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Diverse and Amortised Counterfactual Explanations for Uncertainty Estimates »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Quantization of Bayesian neural networks and its effect on quality of uncertainty »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Class-Distribution-Aware Calibration for Long-Tailed Visual Recognition »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Bayesian Neural Networks with Soft Evidence »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Anomaly Detection for Event Data with Temporal Point Processes »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Batch Inverse-Variance Weighting: Deep Heteroscedastic Regression »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : An Empirical Study of Invariant Risk Minimization on Deep Models »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : A Bayesian Approach to Invariant Deep Neural Networks »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Practical posterior Laplace approximation with optimization-driven second moment estimation »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Variational Generative Flows for Reconstruction Uncertainty Estimation »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Improving the Accuracy-Robustness Trade-Off for Dual-Domain Adversarial Training »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Consistency Regularization Can Improve Robustness to Label Noise »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Neural Variational Gradient Descent »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Evaluating the Use of Reconstruction Error for Novelty Localization »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Accuracy on the Line: on the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : The Hidden Uncertainty in a Neural Network’s Activations »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : On the Calibration of Deterministic Epistemic Uncertainty »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Objective Robustness in Deep Reinforcement Learning »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Epistemic Uncertainty in Learning Chaotic Dynamical Systems »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Towards Stochastic Neural Networks via Inductive Wasserstein Embeddings »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Distribution-free uncertainty quantification for classification under label shift »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : How does a Neural Network's Architecture Impact its Robustness to Noisy Labels? »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Top-label calibration »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Learning to Align the Support of Distributions »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Beyond First-Order Uncertainty Estimation with Evidential Models for Open-World Recognition »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Revisiting Out-of-Distribution Detection: A Simple Baseline is Surprisingly Effective »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Contrastive Predictive Coding for Anomaly Detection and Segmentation »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Multi-headed Neural Ensemble Search »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : A variational approximate posterior for the deep Wishart process »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : What Are Effective Labels for Augmented Data? Improving Calibration and Robustness with AutoLabel »
Yao Qin · Jasper Snoek · Balaji Lakshminarayanan -
2021 : On Stein Variational Neural Network Ensembles »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Uncertainty-Aware Boosted Ensembling in Multi-Modal Settings »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : RouBL: A computationally cheap way to go beyond mean-field variational inference »
Sahar Karimi · Balaji Lakshminarayanan · Jasper Snoek -
2021 : No True State-of-the-Art? OOD Detection Methods are Inconsistent across Datasets »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Out-of-Distribution Generalization with Deep Equilibrium Models »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Mixture Proportion Estimation and PU Learning: A Modern Approach »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : On The Dark Side Of Calibration For Modern Neural Networks »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Domain Adaptation with Factorizable Joint Shift »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Scaling Laws for the Out-of-Distribution Generalization of Image Classifiers »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Learning Invariant Weights in Neural Networks »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Relational Deep Reinforcement Learning and Latent Goals for Following Instructions in Temporal Logic »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : On the Effectiveness of Mode Exploration in Bayesian Model Averaging for Neural Networks »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Training-Free Uncertainty Estimation for Dense Regression: Sensitivity as a Surrogate »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Detecting OODs as datapoints with High Uncertainty »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Multi-task Transformation Learning for Robust Out-of-Distribution Detection »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Directly Training Joint Energy-Based Models for Conditional Synthesis and Calibrated Prediction of Multi-Attribute Data »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Deep Learning with Quantified Uncertainty for Free Electron Laser Scientific Facilities »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : On the reversed bias-variance tradeoff in deep ensembles »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Robust Generalization of Quadratic Neural Networks via Function Identification »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Exploring Corruption Robustness: Inductive Biases in Vision Transformers and MLP-Mixers »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Deep Random Projection Outlyingness for Unsupervised Anomaly Detection »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Deep Deterministic Uncertainty for Semantic Segmentation »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Identifying Invariant and Sparse Predictors in High-dimensional Data »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : On Misclassification-Aware Smoothing for Robustness and Uncertainty Calibration »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : On Pitfalls in OoD Detection: Entropy Considered Harmful »
Andreas Kirsch · Jasper Snoek · Balaji Lakshminarayanan -
2021 : PnPOOD : Out-Of-Distribution Detection for Text Classification via Plug andPlay Data Augmentation »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Augmented Invariant Regularization »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Improved Adversarial Robustness via Uncertainty Targeted Attacks »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Notes on the Behavior of MC Dropout »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Distribution-free Risk-controlling Prediction Sets »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Stochastic Bouncy Particle Sampler for Bayesian Neural Networks »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Novelty detection using ensembles with regularized disagreement »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : A Tale Of Two Long Tails »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Defending against Adversarial Patches with Robust Self-Attention »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Intrinsic uncertainties and where to find them »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Dataset to Dataspace: A Topological-Framework to Improve Analysis of Machine Learning Model Performance »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Analyzing And Improving Neural Networks By Generating Semantic Counterexamples Through Differentiable Rendering »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Thinkback: Task-Specific Out-of-Distribution Detection »
Jasper Snoek · Balaji Lakshminarayanan -
2021 : Relating Adversarially Robust Generalization to Flat Minima »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : Deep Quantile Aggregation »
Balaji Lakshminarayanan · Jasper Snoek -
2021 : What Are Effective Labels for Augmented Data? Improving Calibration and Robustness with AutoLabel »
Yao Qin · Jasper Snoek -
2022 : Plex: Towards Reliability using Pretrained Large Model Extensions »
Dustin Tran · Andreas Kirsch · Balaji Lakshminarayanan · Huiyi Hu · Du Phan · D. Sculley · Jasper Snoek · Jeremiah Liu · Jie Ren · Joost van Amersfoort · Kehang Han · E. Kelly Buchanan · Kevin Murphy · Mark Collier · Mike Dusenberry · Neil Band · Nithum Thain · Rodolphe Jenatton · Tim G. J Rudner · Yarin Gal · Zachary Nado · Zelda Mariet · Zi Wang · Zoubin Ghahramani -
2022 : Plex: Towards Reliability using Pretrained Large Model Extensions »
Dustin Tran · Andreas Kirsch · Balaji Lakshminarayanan · Huiyi Hu · Du Phan · D. Sculley · Jasper Snoek · Jeremiah Liu · JIE REN · Joost van Amersfoort · Kehang Han · Estefany Kelly Buchanan · Kevin Murphy · Mark Collier · Michael Dusenberry · Neil Band · Nithum Thain · Rodolphe Jenatton · Tim G. J Rudner · Yarin Gal · Zachary Nado · Zelda Mariet · Zi Wang · Zoubin Ghahramani -
2023 : Building One-class Detector for Anything: Open-vocabulary Zero-shot OOD Detection Using Text-image Models »
Yunhao Ge · Jie Ren · Jiaping Zhao · Kaifeng Chen · Andrew Gallagher · Laurent Itti · Balaji Lakshminarayanan -
2023 : Morse Neural Networks for Uncertainty Quantification »
Benoit Dherin · Huiyi Hu · JIE REN · Michael Dusenberry · Balaji Lakshminarayanan -
2023 Poster: Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the Machiavelli Benchmark »
Alexander Pan · Jun Shern Chan · Andy Zou · Nathaniel Li · Steven Basart · Thomas Woodside · Hanlin Zhang · Scott Emmons · Dan Hendrycks -
2023 Poster: A Simple Zero-shot Prompt Weighting Technique to Improve Prompt Ensembling in Text-Image Models »
James Allingham · JIE REN · Michael Dusenberry · Xiuye Gu · Yin Cui · Dustin Tran · Jeremiah Liu · Balaji Lakshminarayanan -
2023 Oral: Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the Machiavelli Benchmark »
Alexander Pan · Jun Shern Chan · Andy Zou · Nathaniel Li · Steven Basart · Thomas Woodside · Hanlin Zhang · Scott Emmons · Dan Hendrycks -
2022 : Plex: Towards Reliability using Pretrained Large Model Extensions »
Dustin Tran · Andreas Kirsch · Balaji Lakshminarayanan · Huiyi Hu · Du Phan · D. Sculley · Jasper Snoek · Jeremiah Liu · JIE REN · Joost van Amersfoort · Kehang Han · Estefany Kelly Buchanan · Kevin Murphy · Mark Collier · Michael Dusenberry · Neil Band · Nithum Thain · Rodolphe Jenatton · Tim G. J Rudner · Yarin Gal · Zachary Nado · Zelda Mariet · Zi Wang · Zoubin Ghahramani -
2022 Poster: Scaling Out-of-Distribution Detection for Real-World Settings »
Dan Hendrycks · Steven Basart · Mantas Mazeika · Andy Zou · joseph kwon · Mohammadreza Mostajabi · Jacob Steinhardt · Dawn Song -
2022 Spotlight: Scaling Out-of-Distribution Detection for Real-World Settings »
Dan Hendrycks · Steven Basart · Mantas Mazeika · Andy Zou · joseph kwon · Mohammadreza Mostajabi · Jacob Steinhardt · Dawn Song -
2021 Workshop: A Blessing in Disguise: The Prospects and Perils of Adversarial Machine Learning »
Hang Su · Yinpeng Dong · Tianyu Pang · Eric Wong · Zico Kolter · Shuo Feng · Bo Li · Henry Liu · Dan Hendrycks · Francesco Croce · Leslie Rice · Tian Tian -
2021 : Live Panel Discussion »
Thomas Dietterich · Chelsea Finn · Kamalika Chaudhuri · Yarin Gal · Uri Shalit -
2021 : RL Foundation Panel »
Matthew Botvinick · Thomas Dietterich · Leslie Kaelbling · John Langford · Warrren B Powell · Csaba Szepesvari · Lihong Li · Yuxi Li -
2021 Workshop: Uncertainty and Robustness in Deep Learning »
Balaji Lakshminarayanan · Dan Hendrycks · Sharon Li · Jasper Snoek · Silvia Chiappa · Sebastian Nowozin · Thomas Dietterich -
2021 : Welcome »
Balaji Lakshminarayanan -
2020 : Opening Remarks »
Sharon Yixuan Li -
2020 Poster: Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors »
Mike Dusenberry · Ghassen Jerfel · Yeming Wen · Yian Ma · Jasper Snoek · Katherine Heller · Balaji Lakshminarayanan · Dustin Tran -
2019 : Panel Discussion (moderator: Tom Dietterich) »
Max Welling · Kilian Weinberger · Terrance Boult · Dawn Song · Thomas Dietterich -
2019 Workshop: Uncertainty and Robustness in Deep Learning »
Sharon Yixuan Li · Dan Hendrycks · Thomas Dietterich · Balaji Lakshminarayanan · Justin Gilmer -
2019 : Welcome »
Sharon Yixuan Li -
2019 Poster: Learning from Delayed Outcomes via Proxies with Applications to Recommender Systems »
Timothy Mann · Sven Gowal · Andras Gyorgy · Huiyi Hu · Ray Jiang · Balaji Lakshminarayanan · Prav Srinivasan -
2019 Oral: Learning from Delayed Outcomes via Proxies with Applications to Recommender Systems »
Timothy Mann · Sven Gowal · Andras Gyorgy · Huiyi Hu · Ray Jiang · Balaji Lakshminarayanan · Prav Srinivasan -
2019 Oral: Hybrid Models with Deep and Invertible Features »
Eric Nalisnick · Akihiro Matsukawa · Yee-Whye Teh · Dilan Gorur · Balaji Lakshminarayanan -
2019 Poster: Hybrid Models with Deep and Invertible Features »
Eric Nalisnick · Akihiro Matsukawa · Yee-Whye Teh · Dilan Gorur · Balaji Lakshminarayanan -
2019 Poster: Using Pre-Training Can Improve Model Robustness and Uncertainty »
Dan Hendrycks · Kimin Lee · Mantas Mazeika -
2019 Oral: Using Pre-Training Can Improve Model Robustness and Uncertainty »
Dan Hendrycks · Kimin Lee · Mantas Mazeika -
2018 Poster: Discovering and Removing Exogenous State Variables and Rewards for Reinforcement Learning »
Thomas Dietterich · George Trimponias · Zhitang Chen -
2018 Poster: Open Category Detection with PAC Guarantees »
Si Liu · Risheek Garrepalli · Thomas Dietterich · Alan Fern · Dan Hendrycks -
2018 Oral: Discovering and Removing Exogenous State Variables and Rewards for Reinforcement Learning »
Thomas Dietterich · George Trimponias · Zhitang Chen -
2018 Oral: Open Category Detection with PAC Guarantees »
Si Liu · Risheek Garrepalli · Thomas Dietterich · Alan Fern · Dan Hendrycks -
2017 Workshop: Implicit Generative Models »
Rajesh Ranganath · Ian Goodfellow · Dustin Tran · David Blei · Balaji Lakshminarayanan · Shakir Mohamed