Motion Information Propagation for Neural Video Compression
Authors:
- 1 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6111-6120
Abstract
In most existing neural video codecs, the information flow therein is uni-directional, where only motion coding provides motion vectors for frame coding. In this paper, we argue that, through information interactions, the synergy between motion coding and frame coding can be achieved. We effectively introduce bi-directional information interactions between motion coding and frame coding via our Motion Information Propagation. When generating the temporal contexts for frame coding, the high-dimension motion feature from the motion decoder serves as motion guidance to mitigate the alignment errors. Meanwhile, besides assisting frame coding at the current time step, the feature from context generation will be propagated as motion condition when coding the subsequent motion latent. Through the cycle of such interactions, feature propagation on motion coding is built, strengthening the capacity of exploiting long-range temporal correlation. In addition, we propose hybrid context generation to exploit the multiscale context features and provide better motion condition. Experiments show that our method can achieve 12.9% bit rate saving over the previous SOTA neural video codec.Keywords
This publication has 29 references indexed in Scilit:
- Neural Inter-Frame Compression for Video Coding2019
- Learned Video Compression2019
- Video Compression With Rate-Distortion Autoencoders2019
- DVC: An End-To-End Deep Video Compression Framework2019
- Video Enhancement with Task-Oriented FlowInternational Journal of Computer Vision, 2019
- Deformable Convolutional Networks2017
- MCL-JCV: A JND-based H.264/AVC video quality assessment dataset2016
- Overview of the High Efficiency Video Coding (HEVC) StandardIEEE Transactions on Circuits and Systems for Video Technology, 2012
- Multiscale structural similarity for image quality assessment2004
- Overview of the H.264/AVC video coding standardIEEE Transactions on Circuits and Systems for Video Technology, 2003