Timezone: »

 
Poster
VideoOneNet: Bidirectional Convolutional Recurrent OneNet with Trainable Data Steps for Video Processing
Zoltán Á. Milacski · Barnabás Póczos · Andras Lorincz

Wed Jul 15 02:00 PM -- 02:45 PM & Thu Jul 16 01:00 AM -- 01:45 AM (PDT) @ Virtual #None
in Poster Session 28 »

Deep Neural Networks (DNNs) achieve the state-of-the-art results on a wide range of image processing tasks, however, the majority of such solutions are problem-specific, like most AI algorithms. The One Network to Solve Them All (OneNet) procedure has been suggested to resolve this issue by exploiting a DNN as the proximal operator in Alternating Direction Method of Multipliers (ADMM) solvers for various imaging problems. In this work, we make two contributions, both facilitating end-to-end learning using backpropagation. First, we generalize OneNet to videos by augmenting its convolutional prior network with bidirectional recurrent connections; second, we extend the fixed fully connected linear ADMM data step with another trainable bidirectional convolutional recurrent network. In our computational experiments on the Rotated MNIST, Scanned CIFAR-10 and UCF-101 data sets, the proposed modifications improve performance by a large margin compared to end-to-end convolutional OneNet and 3D Wavelet sparsity on several video processing problems: pixelwise inpainting-denoising, blockwise inpainting, scattered inpainting, super resolution, compressive sensing, deblurring, frame interpolation, frame prediction and colorization. Our two contributions are complementary, and using them together yields the best results.

Keywords: Architectures · Deep Sequence Models · Sparsity and Compressed Sensing · Transfer and Multitask Learning · Sequential, Network, and Time-Series Modeling

Author Information

Zoltán Á. Milacski (ELTE Eötvös Loránd University)
Barnabás Póczos (CMU)
Andras Lorincz (Eotvos Lorand University)

More from the Same Authors