Timezone: »
Many computer vision applications require solving multiple tasks in real-time. A neural network can be trained to solve multiple tasks simultaneously using multi-task learning. This can save computation at inference time as only a single network needs to be evaluated. Unfortunately, this often leads to inferior overall performance as task objectives can compete, which consequently poses the question: which tasks should and should not be learned together in one network when employing multi-task learning? We study task cooperation and competition in several different learning settings and propose a framework for assigning tasks to a few neural networks such that cooperating tasks are computed by the same neural network, while competing tasks are computed by different networks. Our framework offers a time-accuracy trade-off and can produce better accuracy using less inference time than not only a single large multi-task neural network but also many single-task networks.
Author Information
Trevor Standley (Stanford University)
Amir Zamir (Swiss Federal Institute of Technology (EPFL))
Dawn Chen (Google)
Leonidas Guibas (Stanford University)
Jitendra Malik (University of California at Berkeley)
Silvio Savarese (Stanford University)
More from the Same Authors
-
2022 : 3D Common Corruptions for Object Recognition »
Oguzhan Fatih Kar · Teresa Yeo · Amir Zamir -
2022 : 3D Common Corruptions for Object Recognition »
Oguzhan Fatih Kar · Teresa Yeo · Amir Zamir -
2022 Poster: Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging »
Anastasios Angelopoulos · Amit Pal Kohli · Stephen Bates · Michael Jordan · Jitendra Malik · Thayer Alshaabi · Srigokul Upadhyayula · Yaniv Romano -
2022 Spotlight: Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging »
Anastasios Angelopoulos · Amit Pal Kohli · Stephen Bates · Michael Jordan · Jitendra Malik · Thayer Alshaabi · Srigokul Upadhyayula · Yaniv Romano -
2021 Poster: Differentiable Spatial Planning using Transformers »
Devendra Singh Chaplot · Deepak Pathak · Jitendra Malik -
2021 Spotlight: Differentiable Spatial Planning using Transformers »
Devendra Singh Chaplot · Deepak Pathak · Jitendra Malik -
2020 Poster: Goal-Aware Prediction: Learning to Model What Matters »
Suraj Nair · Silvio Savarese · Chelsea Finn -
2020 Poster: Deep Isometric Learning for Visual Recognition »
Haozhi Qi · Chong You · Xiaolong Wang · Yi Ma · Jitendra Malik -
2018 Poster: Learning Representations and Generative Models for 3D Point Clouds »
Panagiotis Achlioptas · Olga Diamanti · Ioannis Mitliagkas · Leonidas Guibas -
2018 Oral: Learning Representations and Generative Models for 3D Point Clouds »
Panagiotis Achlioptas · Olga Diamanti · Ioannis Mitliagkas · Leonidas Guibas -
2017 Workshop: ICML Workshop on Machine Learning for Autonomous Vehicles 2017 »
Li Erran Li · Raquel Urtasun · Andrew Gray · Silvio Savarese -
2017 Poster: Fast k-Nearest Neighbour Search via Prioritized DCI »
Ke Li · Jitendra Malik -
2017 Talk: Fast k-Nearest Neighbour Search via Prioritized DCI »
Ke Li · Jitendra Malik