Low Complexity Neural Network-Based In-loop Filtering with Decomposed Split Luma-Chroma Model for Video Compression

In this paper, a novel low complexity split luma-chroma model is proposed for in-loop filtering in video compression. The basic block of the model adopts the decomposed regular 3x3 convolutional layer, which is replaced by 1x1 pointwise convolutions and 3x1/1x3 separable convolutions via CP decomposition to reduce complexity. It’s further proposed to fuse the two adjacent 1x1 convolutional layers into one. To efficiently exploit the dependencies between luma and chroma while modeling the independent characteristics of luma/chroma component, a novel split luma-chroma architecture within one CNN model is proposed. The input layer and the first hidden layers serving as the common path jointly process luma-chroma inputs. Then the output feature maps are split into luma and chroma feature maps, and they are independently processed using the same basic block as in common path, i.e., one luma path with 24 channels and one chroma path with 8 channels. Experimental results show that the model has 5.66% BD-Rate luma gain over NNVC-4.0 under RA while the chroma gains are also greatly improves, at the complexity of 17.7 kMAC/Pixel. The BD-Rate and kMAC/Pixel plot also shows the superior trade-off between complexity and coding gain compared to state-of-the-art filters. And the subjective results demonstrate improved visual quality. Moreover, the split luma-chroma architecture also possess the flexibility to get arbitrary luma-chroma rate-distortion distribution by adjusting the number of channels in each path.