Adaptive Prediction Structure for Learned Video Compression
Open Access
- 30 December 2024
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 21 (2) , 1-23
Abstract
Learned video compression has developed rapidly and shown competitive rate-distortion performance compared with the latest traditional video coding standard H.266 (VVC). However, existing works were restricted to fixed prediction direction and GoP size. The inflexibility on prediction structure hinders learned video compression towards optimal compression efficiency in diverse motion scenarios. In this article, we propose to advance learned video compression with adaptive prediction structure decision. Specifically, we propose a unified compression framework that supports both forward prediction and bi-directional prediction. The framework can flexibly switch to different prediction direction to achieve better prediction performance. Meanwhile, we propose a low-complexity prediction structure decision algorithm, where prediction direction and GoP size are adaptively determined based on motion complexity to achieve optimal compression efficiency. Experimental results demonstrate that the proposed unified framework with adaptive decision algorithm improves compression efficiency of pure forward prediction-based or bi-directional prediction-based framework with neglectable ( \(0.9\%\) ) encoding time increment. Meanwhile, it achieves comparable compression performance with VVC and recent learned video coding methods.
Keywords
Funding Information
- Key Research and Development Program of Peng Cheng Laboratory (PCL2024A02)
- Shenzhen Science and Technology Program-Shenzhen Cultivation of Excellent Scientific and Technological Innovation Talents (RCJC20200714114435057)
- Shenzhen Science and Technology Program-Shenzhen Hong Kong (SGDX20211123144400001)
- National Natural Science Foundation of China (U21B2012)
This publication has 36 references indexed in Scilit:
- B-CANF: Adaptive B-Frame Coding With Conditional Augmented Normalizing FlowsIEEE Transactions on Circuits and Systems for Video Technology, 2023
- DMVC: Decomposed Motion Modeling for Learned Video CompressionIEEE Transactions on Circuits and Systems for Video Technology, 2022
- Advancing Learned Video Compression With In-Loop Frame PredictionIEEE Transactions on Circuits and Systems for Video Technology, 2022
- End-to-End Neural Video Coding Using a Compound Spatiotemporal RepresentationIEEE Transactions on Circuits and Systems for Video Technology, 2022
- End-to-End Rate-Distortion Optimized Learned Hierarchical Bi-Directional Video CompressionIEEE Transactions on Image Processing, 2021
- Overview of the Versatile Video Coding (VVC) Standard and its ApplicationsIEEE Transactions on Circuits and Systems for Video Technology, 2021
- Learning for Video Compression With Recurrent Auto-Encoder and Recurrent Probability ModelIEEE Journal of Selected Topics in Signal Processing, 2020
- Video Enhancement with Task-Oriented FlowInternational Journal of Computer Vision, 2019
- Overview of the High Efficiency Video Coding (HEVC) StandardIEEE Transactions on Circuits and Systems for Video Technology, 2012
- Overview of the H.264/AVC video coding standardIEEE Transactions on Circuits and Systems for Video Technology, 2003