Timezone: »

 
Oral
Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles
Chaitanya Ryali · Yuan-Ting Hu · Daniel Bolya · Chen Wei · Haoqi Fan · Po-Yao Huang · Vaibhav Aggarwal · Arkabandhu Chowdhury · Omid Poursaeed · Judy Hoffman · Jitendra Malik · Yanghao Li · Christoph Feichtenhofer

Tue Jul 25 09:18 PM -- 09:26 PM (PDT) @ Meeting Room 313
in Oral A2 Computer Vision and Efficient ML »

Modern hierarchical vision transformers have added several vision-specific components in the pursuit of supervised classification performance. While these components lead to effective accuracies and attractive FLOP counts, the added complexity actually makes these transformers slower than their vanilla ViT counterparts. In this paper, we argue that this additional bulk is unnecessary. By pretraining with a strong visual pretext task (MAE), we can strip out all the bells-and-whistles from a state-of-the-art multi-stage vision transformer without losing accuracy. In the process, we create Hiera, an extremely simple hierarchical vision transformer that is more accurate than previous models while being significantly faster both at inference and during training. We evaluate Hiera on a variety of tasks for image and video recognition. Our code and models are available at https://github.com/facebookresearch/hiera.

[ PDF] 

Author Information

Chaitanya Ryali (FAIR, Meta AI)
Yuan-Ting Hu (UIUC)
Daniel Bolya (Georgia Tech Meta)
Chen Wei (Johns Hopkins University)
Haoqi Fan (Facebook AI Research)
Po-Yao Huang (Facebook)
Vaibhav Aggarwal (Facebook)
Arkabandhu Chowdhury (Facebook)
Omid Poursaeed (Meta AI)
Judy Hoffman (Georgia Institute of Technology)
Jitendra Malik (University of California at Berkeley)
Yanghao Li
Christoph Feichtenhofer (Facebook)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors