Timezone: »

 
Poster
RankMe: Assessing the Downstream Performance of Pretrained Self-Supervised Representations by Their Rank
Quentin Garrido · Randall Balestriero · Laurent Najman · Yann LeCun

Thu Jul 27 04:30 PM -- 06:00 PM (PDT) @ Exhibit Hall 1 #609
in Poster Session 6 »

Joint-Embedding Self Supervised Learning (JE-SSL) has seen a rapid development, with the emergence of many method variations but only few principled guidelines that would help practitioners to successfully deploy them. The main reason for that pitfall comes from JE-SSL's core principle of not employing any input reconstruction therefore lacking visual cues of unsuccessful training. Adding non informative loss values to that, it becomes difficult to deploy SSL on a new dataset for which no labels can help to judge the quality of the learned representation. In this study, we develop a simple unsupervised criterion that is indicative of the quality of the learned JE-SSL representations: their effective rank. Albeit simple and computationally friendly, this method ---coined RankMe--- allows one to assess the performance of JE-SSL representations, even on different downstream datasets, without requiring any labels. A further benefit of RankMe is that it does not have any training or hyper-parameters to tune. Through thorough empirical experiments involving hundreds of training episodes, we demonstrate how RankMe can be used for hyperparameter selection with nearly no reduction in final performance compared to the current selection method that involve a dataset's labels. We hope that RankMe will facilitate the deployment of JE-SSL towards domains that do not have the opportunity to rely on labels for representations' quality assessment.

[ Poster] 

Author Information

Quentin Garrido (Meta AI - FAIR, Université Gustave Eiffel)
Randall Balestriero (Rice University)
Laurent Najman (Université Gustave Eiffel - ESIEE Paris)
Laurent Najman

Laurent Najman received the Habilitation à Diriger les Recherches in 2006 from the University of Marne-la-Vallée, a Ph.D. in applied mathematics from Paris-Dauphine University in 1994 with the highest honor (Félicitations du Jury) and an “Ingénieur” degree from the Ecole des Mines de Paris in 1991. After earning his engineering degree, he worked in the Central Research Laboratories of Thomson-CSF for three years, working on some problems of infrared image segmentation using mathematical morphology. He then joined a start-up company named Animation Science in 1995, as director of research and development. The technology of particle systems for computer graphics and scientific visualization, developed by the company under his technical leadership received several awards, including the “European Information Technology Prize 1997” awarded by the European Commission (Esprit program) and by the European Council for Applied Science and Engineering and the “Hottest Products of the Year 1996” awarded by the Computer Graphics World journal. In 1998, he joined OCÉ Print Logic Technologies, as senior scientist. He worked there on various problem of image analysis dedicated to scanning and printing. In 2002, he joined the Computer Sciences Department of ESIEE, Paris, where he is full professor and the leader of the A3SI team of the Laboratoire d’Informatique Gaspard Monge, Université Gustave Eiffel. His current research interests include the study of the topology of discrete structures (such as graphs, hierarchies, and simplicial complexes), using discrete mathematical morphology and discrete optimization.

Yann LeCun (New York University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors