Timezone: »

 
Poster
Flow-Guided Sparse Transformer for Video Deblurring
Jing Lin · Yuanhao Cai · Xiaowan Hu · Haoqian Wang · Youliang Yan · Xueyi Zou · Henghui Ding · Yulun Zhang · Radu Timofte · Luc Van Gool

Thu Jul 21 03:00 PM -- 05:00 PM (PDT) @ Hall E #214
in Poster Session 3 »
Exploiting similar and sharper scene patches in spatio-temporal neighborhoods is critical for video deblurring. However, CNN-based methods show limitations in capturing long-range dependencies and modeling non-local self-similarity. In this paper, we propose a novel framework, Flow-Guided Sparse Transformer (FGST), for video deblurring. In FGST, we customize a self-attention module, Flow-Guided Sparse Window-based Multi-head Self-Attention (FGSW-MSA). For each $query$ element on the blurry reference frame, FGSW-MSA enjoys the guidance of the estimated optical flow to globally sample spatially sparse yet highly related $key$ elements corresponding to the same scene patch in neighboring frames. Besides, we present a Recurrent Embedding (RE) mechanism to transfer information from past frames and strengthen long-range temporal dependencies. Comprehensive experiments demonstrate that our proposed FGST outperforms state-of-the-art (SOTA) methods on both DVD and GOPRO datasets and yields visually pleasant results in real video deblurring. https://github.com/linjing7/VR-Baseline
Keywords: APP: Computer Vision · DL: Attention Mechanisms · DL: Generative Models and Autoencoders
[ Slides]  [ Poster]  [ Paper PDF] 

Author Information

Jing Lin (Tsinghua Univisity, Tsinghua Shenzhen International Graduate School)
Yuanhao Cai (Tsinghua Univisity, Tsinghua Shenzhen International Graduate School)
Xiaowan Hu (Tsinghua Univisity, Tsinghua Shenzhen International Graduate School)
Haoqian Wang (Tsinghua Shenzhen International Graduate School, Tsinghua University)
Youliang Yan (Huawei Noah's Ark Lab)
Xueyi Zou (Huawei Noah's Ark Lab)
Henghui Ding (ETH Zurich)
Yulun Zhang (ETH Zurich)

I am a postdoctoral researcher at Computer Vision Lab, ETH Zürich, Switzerland, working with Prof. Luc Van Gool. Previously, I obtained my PhD degree at Department of Electrical & Computer Engineering, Northeastern University, USA, in Aug. 2021. Before that I received my master degree in the Department of Automation, Tsinghua University, China, in Jul. 2017 and B.E degree from School of Electronic Engineering, Xidian University, China, in Jul. 2013. My research interest broadly includes machine learning and computer vision. Specifically, I focus on image/video restoration (e.g., super-resolution, denoising, deblurring), synthesis (e.g., style transfer, texture transfer), biomedical image enhancement and analysis,deep model compression, computational imaging (e.g., spectral compressive imaging), etc.

Radu Timofte
Luc Van Gool (ETH Zurich)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors