Timezone: »
Tutorial
Self-Attention for Computer Vision
Aravind Srinivas · Prajit Ramachandran · Ashish Vaswani
Mon Jul 19 08:00 PM -- 10:45 PM (PDT) @ Virtual
The tutorial will be about the application of self-attention mechanisms in computer vision. Self-Attention has been widely adopted in NLP, with the fully attentional Transformer model having largely replaced RNNs and now being used in state-of-the-art language understanding models like GPT, BERT, XLNet, T5, Electra, and Meena. Thus, there has been a tremendous interest in studying whether self-attention can have a similarly big and far-reaching impact in computer vision. However, vision tasks have different properties compared to language tasks, so a lot of research has been devoted to exploring the best way to apply self-attention to visual models. This tutorial will cover many of the different applications of self-attention in vision in order to give the viewer a broad and precise understanding of this subfield.
|
Mon 8:00 p.m. - 8:45 p.m.
|
Self-Attention for Computer Vision
(
talk
)
SlidesLive Video » |
Ashish Vaswani 🔗 |
|
Mon 8:45 p.m. - 9:00 p.m.
|
Self-Attention for Computer Vision
|
🔗 |
|
Mon 9:00 p.m. - 9:45 p.m.
|
Self-Attention for Computer Vision
(
talk
)
SlidesLive Video » |
Prajit Ramachandran 🔗 |
|
Mon 9:45 p.m. - 10:00 p.m.
|
Self-Attention for Computer Vision
|
🔗 |
|
Mon 10:00 p.m. - 10:45 p.m.
|
Self-Attention for Computer Vision
(
talk
)
SlidesLive Video » |
Aravind Srinivas 🔗 |
Author Information
Aravind Srinivas (UC Berkeley)
Prajit Ramachandran (Google)
Ashish Vaswani (Google Brain)
More from the Same Authors
-
2021 Workshop: ICML 2021 Workshop on Unsupervised Reinforcement Learning »
Feryal Behbahani · Joelle Pineau · Lerrel Pinto · Roberta Raileanu · Aravind Srinivas · Denis Yarats · Amy Zhang -
2021 Poster: SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learning »
Kimin Lee · Michael Laskin · Aravind Srinivas · Pieter Abbeel -
2021 Spotlight: SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learning »
Kimin Lee · Michael Laskin · Aravind Srinivas · Pieter Abbeel -
2021 : Self-Attention for Computer Vision »
Aravind Srinivas -
2021 : Self-Attention for Computer Vision »
Prajit Ramachandran -
2021 : Self-Attention for Computer Vision »
Ashish Vaswani -
2021 Tutorial: Unsupervised Learning for Reinforcement Learning »
Aravind Srinivas · Pieter Abbeel -
2021 : Part 1: Representation Learning in RL »
Aravind Srinivas -
2020 Poster: CURL: Contrastive Unsupervised Representations for Reinforcement Learning »
Michael Laskin · Aravind Srinivas · Pieter Abbeel -
2020 Poster: Revisiting Fundamentals of Experience Replay »
William Fedus · Prajit Ramachandran · Rishabh Agarwal · Yoshua Bengio · Hugo Larochelle · Mark Rowland · Will Dabney -
2020 Poster: Revisiting Spatial Invariance with Low-Rank Local Connectivity »
Gamaleldin Elsayed · Prajit Ramachandran · Jon Shlens · Simon Kornblith -
2019 Poster: Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design »
Jonathan Ho · Peter Chen · Aravind Srinivas · Rocky Duan · Pieter Abbeel -
2019 Oral: Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design »
Jonathan Ho · Peter Chen · Aravind Srinivas · Rocky Duan · Pieter Abbeel -
2018 Poster: Image Transformer »
Niki Parmar · Ashish Vaswani · Jakob Uszkoreit · Lukasz Kaiser · Noam Shazeer · Alexander Ku · Dustin Tran -
2018 Poster: Fast Decoding in Sequence Models Using Discrete Latent Variables »
Lukasz Kaiser · Samy Bengio · Aurko Roy · Ashish Vaswani · Niki Parmar · Jakob Uszkoreit · Noam Shazeer -
2018 Oral: Image Transformer »
Niki Parmar · Ashish Vaswani · Jakob Uszkoreit · Lukasz Kaiser · Noam Shazeer · Alexander Ku · Dustin Tran -
2018 Oral: Fast Decoding in Sequence Models Using Discrete Latent Variables »
Lukasz Kaiser · Samy Bengio · Aurko Roy · Ashish Vaswani · Niki Parmar · Jakob Uszkoreit · Noam Shazeer -
2018 Poster: Universal Planning Networks: Learning Generalizable Representations for Visuomotor Control »
Aravind Srinivas · Allan Jabri · Pieter Abbeel · Sergey Levine · Chelsea Finn -
2018 Oral: Universal Planning Networks: Learning Generalizable Representations for Visuomotor Control »
Aravind Srinivas · Allan Jabri · Pieter Abbeel · Sergey Levine · Chelsea Finn