Learned Rate-Distortion Cost Prediction for Ultrafast Screen Content Intra Coding
- 18 July 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (3) , 1976-1980
Abstract
As online collaborations become more prevalent, screen content has become increasingly important in real-time video communications. To reduce communication costs, the H.265/HEVC standard introduced the Screen Content Coding (SCC) extension, which achieves significant bits savings but comes with a higher encoding complexity. There is a need for ultrafast SCC encoding to meet the demands of real-time applications. Our key idea is to predict the rate-distortion (RD) cost of each possible coding unit under each possible mode, rather than performing actual coding to obtain the RD cost. Specifically, we construct neural networks to predict RD costs for intra prediction, palette, and normal intra block copy (IBC) modes. For IBC merge mode, we conduct motion compensation trials and use a linear regression network for prediction. Using the predicted RD costs, we create a partition-mode map set that determines not only block partitioning but also optimal modes, significantly reducing encoding complexity. Our experimental results demonstrate that our method achieves a more than 90% reduction in encoding time with an average 9.4% BD-rate increase compared to the HEVC-SCC reference software in the all-intra configuration.Keywords
Funding Information
- Natural Science Foundation of China (62021001, 62022075, 62032006)
- Fundamental Research Funds for the Central Universities (WK3490000006)
- Graphics Processing Unit (GPU) Cluster Built by Multimedia Computing and Communication (MCC) Laboratory of Information Science and Technology Institution, USTC
This publication has 18 references indexed in Scilit:
- Two-Step Fast Mode Decision for Intra Coding of Screen ContentIEEE Transactions on Circuits and Systems for Video Technology, 2022
- Mode skipping for screen content coding based on Neural Network ClassifierJournal of Real-Time Image Processing, 2021
- Overview of Screen Content Coding in Recently Developed Video Coding StandardsIEEE Transactions on Circuits and Systems for Video Technology, 2021
- An Adaptive Fast Intra Mode Decision in Screen Content Coding2019
- DeepSCC: Deep Learning Based Fast Prediction Network for Screen Content CodingIEEE Transactions on Circuits and Systems for Video Technology, 2019
- Machine Learning-Based Fast Intra Mode Decision for HEVC Screen Content Coding via Decision TreesIEEE Transactions on Circuits and Systems for Video Technology, 2019
- Intra Block Copy in HEVC Screen Content Coding ExtensionsIEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2016
- Overview of the Emerging HEVC Screen Content Coding ExtensionIEEE Transactions on Circuits and Systems for Video Technology, 2015
- Overview of the High Efficiency Video Coding (HEVC) StandardIEEE Transactions on Circuits and Systems for Video Technology, 2012
- Virtualized Screen: A Third Element for Cloud–Mobile ConvergenceIEEE MultiMedia, 2011