DMVC: Decomposed Motion Modeling for Learned Video Compression
- 29 December 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 33 (7) , 3502-3515
Abstract
Inter prediction is the critical component in hybrid coding framework to deal with the temporal redundancy. Most of the neural video coding methods typically follow the motion compensation based inter coding scheme, establishing motion vector (MV) as the central role. In this paper, we innovatively propose an efficient motion modeling approach by inherently decomposing it into two components, the intrinsic motion and the compensatory motion. The intrinsic motion originates from the implicit spatiotemporal context hidden in the historical sequence, which can be intuitively captured free of bits. On the top of it, the compensatory motion acts a role of structural refinement and texture enhancement as a form of side information. In particular, the inter prediction is performed in the feature space as a manner of progressive temporal transition, conditioned on the decomposed motion. By the motion decomposition paradigm, we innovatively answer the question of motion representation, compensation and coding in the learned video compression framework. With the temporal prediction, the remaining pixel residue is signaled to obtain the reconstruction. Extensive experimental results demonstrate that the proposed method achieves state-of-the-art coding performance on par with other end-to-end coding methods, and outperforms versatile video coding (VVC) under low-delay P (LDP) configuration in terms of MS-SSIM metric.Keywords
Funding Information
- National Natural Science Foundation of China (62025101, 62101007, 62031013)
- High performance Computing Platform of Peking University
This publication has 34 references indexed in Scilit:
- One-for-All: Grouped Variation Network-Based Fractional Interpolation in Video CodingIEEE Transactions on Image Processing, 2018
- Deformable Convolutional Networks2017
- Densely Connected Convolutional Networks2017
- Optical Flow Estimation Using a Spatial Pyramid Network2017
- MCL-JCV: A JND-based H.264/AVC video quality assessment dataset2016
- Deep Residual Learning for Image Recognition2016
- End-to-end optimization of nonlinear transform codes for perceptual quality2016
- Overview of the High Efficiency Video Coding (HEVC) StandardIEEE Transactions on Circuits and Systems for Video Technology, 2012
- Overview of the H.264/AVC video coding standardIEEE Transactions on Circuits and Systems for Video Technology, 2003
- Long Short-Term MemoryNeural Computation, 1997