Differentiable bit-rate estimation for neural-based video codec enhancement
Preprint
Abstract
Neural networks (NN) can improve standard video compression by pre- and post-processing the encoded video. For optimal NN training, the standard codec needs to be replaced with a codec proxy that can provide derivatives of estimated bit-rate and distortion, which are used for gradient back-propagation. Since entropy coding of standard codecs is designed to take into account non-linear dependencies between transform coefficients, bit-rates cannot be well approximated with simple per-coefficient estimators. This paper presents a new approach for bit-rate estimation that is similar to the type employed in training end-to-end neural codecs, and able to efficiently take into account those statistical dependencies. It is defined from a mathematical model that provides closed-form formulas for the estimates and their gradients, reducing the computational complexity. Experimental results demonstrate the method's accuracy in estimating HEVC/H.265 codec bit-rates.Keywords
All Related Versions
- Version 1, 2023-01-24, ArXiv
- Published version: , 379.
This publication has 8 references indexed in Scilit:
- Codec-Simulation Network for Joint Optimization of Video Coding With Pre- and Post-ProcessingIEEE Open Journal of Circuits and Systems, 2021
- Advances in Video Compression System Using Deep Neural Network: A Review and Case StudiesProceedings of the IEEE, 2021
- The Rate-Distortion-Accuracy Tradeoff: JPEG Case Study2021
- Image and Video Compression With Neural Networks: A ReviewIEEE Transactions on Circuits and Systems for Video Technology, 2019
- High Throughput CABAC Entropy Coding in HEVCIEEE Transactions on Circuits and Systems for Video Technology, 2012
- The H.264 Advanced Video Compression Standard2010
- Rate Control for H.264 Video With Enhanced Rate and Distortion ModelsIEEE Transactions on Circuits and Systems for Video Technology, 2007
- Rate-distortion analysis for H.264/AVC video coding and its application to rate controlIEEE Transactions on Circuits and Systems for Video Technology, 2005