Timezone: »

 
Spotlight
Multi-Task Learning as a Bargaining Game
Aviv Navon · Aviv Shamsian · Idan Achituve · Haggai Maron · Kenji Kawaguchi · Gal Chechik · Ethan Fetaya

Tue Jul 19 07:30 AM -- 07:35 AM (PDT) @ Room 318 - 320
in Transfer/Multitask/Meta Learning »

In Multi-task learning (MTL), a joint model is trained to simultaneously make predictions for several tasks. Joint training reduces computation costs and improves data efficiency; however, since the gradients of these different tasks may conflict, training a joint model for MTL often yields lower performance than its corresponding single-task counterparts. A common method for alleviating this issue is to combine per-task gradients into a joint update direction using a particular heuristic. In this paper, we propose viewing the gradients combination step as a bargaining game, where tasks negotiate to reach an agreement on a joint direction of parameter update. Under certain assumptions, the bargaining problem has a unique solution, known as the \emph{Nash Bargaining Solution}, which we propose to use as a principled approach to multi-task learning. We describe a new MTL optimization procedure, Nash-MTL, and derive theoretical guarantees for its convergence. Empirically, we show that Nash-MTL achieves state-of-the-art results on multiple MTL benchmarks in various domains.

Keywords: MISC: Transfer, Multitask and Meta-learning

Author Information

Aviv Navon (Bar-Ilan University)
Aviv Shamsian (Bar Ilan University)
Idan Achituve (Bar-Ilan)
Haggai Maron (NVIDIA Research)

I am a Research Scientist at NVIDIA Research. My main fields of interest are machine learning, optimization, and shape analysis. More specifically, I am working on applying deep learning to irregular domains (e.g., graphs, point clouds, and surfaces) and graph/shape matching problems. I completed my Ph.D. in 2019 at the Department of Computer Science and Applied Mathematics at the Weizmann Institute of Science under the supervision of Prof. Yaron Lipman.

Kenji Kawaguchi (National University of Singapore)
Gal Chechik (NVIDIA / Bar-Ilan University)
Ethan Fetaya (Bar-Ilan University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors