Timezone: »
Many computer vision applications require solving multiple tasks in real-time. A neural network can be trained to solve multiple tasks simultaneously using multi-task learning. This can save computation at inference time as only a single network needs to be evaluated. Unfortunately, this often leads to inferior overall performance as task objectives can compete, which consequently poses the question: which tasks should and should not be learned together in one network when employing multi-task learning? We study task cooperation and competition in several different learning settings and propose a framework for assigning tasks to a few neural networks such that cooperating tasks are computed by the same neural network, while competing tasks are computed by different networks. Our framework offers a time-accuracy trade-off and can produce better accuracy using less inference time than not only a single large multi-task neural network but also many single-task networks.
Author Information
Trevor Standley (Stanford University)
Amir Zamir (Swiss Federal Institute of Technology (EPFL))
Dawn Chen (Google)
Leonidas Guibas (Stanford University)
Jitendra Malik (University of California at Berkeley)
Silvio Savarese (Stanford University)
More from the Same Authors
-
2022 : 3D Common Corruptions for Object Recognition »
Oguzhan Fatih Kar · Teresa Yeo · Amir Zamir -
2023 Poster: BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models »
Junnan Li · DONGXU LI · Silvio Savarese · Steven Hoi -
2023 Poster: Towards Learning Geometric Eigen-Lengths Crucial for Fitting Tasks »
Yijia Weng · Kaichun Mo · Ruoxi Shi · Yanchao Yang · Leonidas Guibas -
2023 Poster: Modeling Dynamic Environments with Scene Graph Memory »
Andrey Kurenkov · Michael Lingelbach · Tanmay Agarwal · Emily Jin · Chengshu Li · Ruohan Zhang · Li Fei-Fei · Jiajun Wu · Silvio Savarese · Roberto Martín-Martín -
2023 Poster: Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles »
Chaitanya Ryali · Yuan-Ting Hu · Daniel Bolya · Chen Wei · Haoqi Fan · Po-Yao Huang · Vaibhav Aggarwal · Arkabandhu Chowdhury · Omid Poursaeed · Judy Hoffman · Jitendra Malik · Yanghao Li · Christoph Feichtenhofer -
2023 Oral: Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles »
Chaitanya Ryali · Yuan-Ting Hu · Daniel Bolya · Chen Wei · Haoqi Fan · Po-Yao Huang · Vaibhav Aggarwal · Arkabandhu Chowdhury · Omid Poursaeed · Judy Hoffman · Jitendra Malik · Yanghao Li · Christoph Feichtenhofer -
2022 : 3D Common Corruptions for Object Recognition »
Oguzhan Fatih Kar · Teresa Yeo · Amir Zamir -
2022 Poster: Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging »
Anastasios Angelopoulos · Amit Pal Kohli · Stephen Bates · Michael Jordan · Jitendra Malik · Thayer Alshaabi · Srigokul Upadhyayula · Yaniv Romano -
2022 Spotlight: Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging »
Anastasios Angelopoulos · Amit Pal Kohli · Stephen Bates · Michael Jordan · Jitendra Malik · Thayer Alshaabi · Srigokul Upadhyayula · Yaniv Romano -
2021 Poster: Differentiable Spatial Planning using Transformers »
Devendra Singh Chaplot · Deepak Pathak · Jitendra Malik -
2021 Spotlight: Differentiable Spatial Planning using Transformers »
Devendra Singh Chaplot · Deepak Pathak · Jitendra Malik -
2020 Poster: Goal-Aware Prediction: Learning to Model What Matters »
Suraj Nair · Silvio Savarese · Chelsea Finn -
2020 Poster: Deep Isometric Learning for Visual Recognition »
Haozhi Qi · Chong You · Xiaolong Wang · Yi Ma · Jitendra Malik -
2018 Poster: Learning Representations and Generative Models for 3D Point Clouds »
Panagiotis Achlioptas · Olga Diamanti · Ioannis Mitliagkas · Leonidas Guibas -
2018 Oral: Learning Representations and Generative Models for 3D Point Clouds »
Panagiotis Achlioptas · Olga Diamanti · Ioannis Mitliagkas · Leonidas Guibas -
2017 Workshop: ICML Workshop on Machine Learning for Autonomous Vehicles 2017 »
Li Erran Li · Raquel Urtasun · Andrew Gray · Silvio Savarese -
2017 Poster: Fast k-Nearest Neighbour Search via Prioritized DCI »
Ke Li · Jitendra Malik -
2017 Talk: Fast k-Nearest Neighbour Search via Prioritized DCI »
Ke Li · Jitendra Malik