Neural Network-Based Reference Block Quality Enhancement for Motion Compensation Prediction

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Inter prediction is a crucial part of hybrid video coding frameworks, and it is used to eliminate redundancy in adjacent frames and improve coding performance. During inter prediction, motion estimation is used to find the reference block that is most similar to the current block, and the following motion compensation is used to shift the reference block fractionally to obtain the prediction block. The closer the reference block is to the original block, the higher the coding efficiency is. To improve the quality of reference blocks, a quality enhancement network (RBENN) that is dedicated to reference blocks is proposed. The main body of the network consists of 10 residual modules, with two convolution layers for preprocessing and feature extraction. Each residual module consists of two convolutional layers, one ReLU activation, and a shortcut. The network uses the luma reference block as input before motion compensation, and the enhanced reference block is then filtered by the default fractional interpolation. Moreover, the proposed method can be used for both conventional motion compensation and affine motion compensation. Experimental results showed that RBENN could achieve a −1.35% BD rate on average under the low-delay P (LDP) configuration compared with the latest H.266/VVC.
Funding Information
  • National Natural Science Foundation of China (62222110, 62172259, tsqn202103001, VRLAB2021A01, ZR2022ZD38, YDZX2021002, 2020CXGC010109)
  • Taishan Scholar Project of Shandong Province (62222110, 62172259, tsqn202103001, VRLAB2021A01, ZR2022ZD38, YDZX2021002, 2020CXGC010109)
  • Open Project Program of the State Key Laboratory of Virtual Reality Technology and Systems, Beihang University (62222110, 62172259, tsqn202103001, VRLAB2021A01, ZR2022ZD38, YDZX2021002, 2020CXGC010109)
  • Natural Science Foundation of Shandong Province of China (62222110, 62172259, tsqn202103001, VRLAB2021A01, ZR2022ZD38, YDZX2021002, 2020CXGC010109)
  • Central Guidance Fund for Local Science and Technology Development of Shandong Province (62222110, 62172259, tsqn202103001, VRLAB2021A01, ZR2022ZD38, YDZX2021002, 2020CXGC010109)
  • Major Scientific and Technological Innovation Project of Shandong Province (62222110, 62172259, tsqn202103001, VRLAB2021A01, ZR2022ZD38, YDZX2021002, 2020CXGC010109)

This publication has 23 references indexed in Scilit: