STRANet: Soft-Target and Restriction-Aware Neural Network for Efficient VVC Intra Coding
- 15 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (11) , 11993-12005
Abstract
The Versatile Video Coding (VVC) standard introduces a quad-tree with a nested multi-type tree (QTMT) partition structure to improve the rate-distortion (RD) performance, but this leads to a substantial increase in encoding complexity. Previous studies have labeled partition modes of CUs using hard targets (i.e., one-hot labels) generated by VVC reference software (VTM), which is challenging for neural networks to predict accurately. Furthermore, in earlier works, the VVC restrictions are not incorporated into convolutional neural network (CNN), not fully exploiting the predicting capacity of CNN. In this paper, we propose a novel soft-target and restriction-aware neural network (STRANet) to address these issues. Firstly, inspired by the observation that a CU may split differently under various circumstances, we collect these RD costs and precisely estimate the probability of each partition mode to generate a soft target. Secondly, our neural network incorporates QP and restriction type through attention modules so as to output predictions that are standard-compliant with simple post-processing. Thirdly, Window Attention Module, a combination of CNN and attention mechanism, is adopted to further enhance performance on GPU. Through the application of these methods, STRANet reduces encoding time by 51.84% and 61.00% with 0.44% and 0.84% Bjøntegaard delta bit-rate (BD-BR) increase, superior to state-of-the-art methods. The code has been released at https://github.com/cppppp/STRANet.Keywords
Funding Information
- national natural foundation of China (62071014)
This publication has 48 references indexed in Scilit:
- The Devil Is in the Details: Window-based Attention for Image Compression2022
- An efficient low-complexity block partition scheme for VVC intra codingJournal of Real-Time Image Processing, 2021
- Fast CU partition decision for H.266/VVC based on the improved DAG-SVM classifier modelMultimedia Systems, 2020
- A fast CU size decision algorithm for VVC intra prediction based on support vector machineMultimedia Tools and Applications, 2020
- Gradient-Based Early Termination of CU Partition in VVC Intra Coding2020
- Context-Based Ternary Tree Decision Method in Versatile Video Coding for Fast Intra CodingIEEE Access, 2019
- Fast CTU Partition Decision Algorithm for VVC Intra and Inter Coding2019
- Fast QTBT Partition Algorithm for Intra Frame Coding through Convolutional Neural NetworkIEEE Access, 2018
- Effective Quadtree Plus Binary Tree Block Partition Decision for Future Video Coding2017
- Deep Residual Learning for Image Recognition2016