Timezone: »

 
Poster
Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning
Sébastien Lachapelle · Tristan Deleu · Divyat Mahajan · Ioannis Mitliagkas · Yoshua Bengio · Simon Lacoste-Julien · Quentin Bertrand

Wed Jul 26 05:00 PM -- 06:30 PM (PDT) @ Exhibit Hall 1 #644
in Poster Session 4 »

Although disentangled representations are often said to be beneficial for downstream tasks, current empirical and theoretical understanding is limited. In this work, we provide evidence that disentangled representations coupled with sparse task-specific predictors improve generalization. In the context of multi-task learning, we prove a new identifiability result that provides conditions under which maximally sparse predictors yield disentangled representations. Motivated by this theoretical result, we propose a practical approach to learn disentangled representations based on a sparsity-promoting bi-level optimization problem. Finally, we explore a meta-learning version of this algorithm based on group Lasso multiclass SVM predictors, for which we derive a tractable dual formulation. It obtains competitive results on standard few-shot classification benchmarks, while each task is using only a fraction of the learned representations.

[ Slides]  [ PDF] 

Author Information

Sébastien Lachapelle (Université de Montréal, Mila)
Tristan Deleu (Mila - Université de Montréal)
Divyat Mahajan (Mila – Quebec AI Institute)
Ioannis Mitliagkas (MILA, UdeM)
Yoshua Bengio (Mila - Quebec AI Institute)
Simon Lacoste-Julien (Mila, University of Montreal & Samsung SAIL Montreal)
Simon Lacoste-Julien

Simon Lacoste-Julien is an associate professor at Mila and DIRO from Université de Montréal, and Canada CIFAR AI Chair holder. He also heads part time the SAIT AI Lab Montreal from Samsung. His research interests are machine learning and applied math, with applications in related fields like computer vision and natural language processing. He obtained a B.Sc. in math., physics and computer science from McGill, a PhD in computer science from UC Berkeley and a post-doc from the University of Cambridge. He spent a few years as a research faculty at INRIA and École normale supérieure in Paris before coming back to his roots in Montreal in 2016 to answer the call from Yoshua Bengio in growing the Montreal AI ecosystem.

Quentin Bertrand (Mila)

More from the Same Authors