Timezone: »

Measuring Representational Robustness of Neural Networks Through Shared Invariances
Vedant Nanda · Till Speicher · Camila Kolling · John P Dickerson · Krishna Gummadi · Adrian Weller

Thu Jul 21 01:35 PM -- 01:55 PM (PDT) @ Ballroom 1 & 2

Robustness has emerged as a key consideration in the study of machine learning models, asserting that a model's output is invariant to certain perturbations of its input. One goal is to study the relative robustness of two models, i.e. to assert that one model will not make mistakes on examples that another model (or a human) gets right. Currently, only a few methods are suitable to compare models directly, the most prominent of which are representation similarity metrics such as CKA and SVCCA. However, we demonstrate empirically that these representation similarity metrics cannot be used reliably to make robustness judgements. Based on this insight, we develop a new directional metric to compare the relative robustness of models, that measures how well a target model preserves the invariances of a reference model. We show that our measure retains the desirable properties of previous similarity metrics, but also allows us to make statements about the shared invariance of models. With the help of our measure, we are able to gain insights about how shared invariances vary with changes in weight initialization, architecture, loss, and training dataset.

Author Information

Vedant Nanda (University of Maryland, College Park)
Till Speicher (MPI-SWS)
Camila Kolling (PUCRS)
John P Dickerson (Arthur AI & Univ. of Maryland)
Krishna Gummadi (MPI-SWS)
Adrian Weller (University of Cambridge, Alan Turing Institute)
Adrian Weller

Adrian Weller is Programme Director for AI at The Alan Turing Institute, the UK national institute for data science and AI, and is a Turing AI Fellow leading work on trustworthy Machine Learning (ML). He is a Principal Research Fellow in ML at the University of Cambridge, and at the Leverhulme Centre for the Future of Intelligence where he is Programme Director for Trust and Society. His interests span AI, its commercial applications and helping to ensure beneficial outcomes for society. Previously, Adrian held senior roles in finance. He received a PhD in computer science from Columbia University, and an undergraduate degree in mathematics from Trinity College, Cambridge.

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors