Multi-frame Quality Enhancement for Compressed Video
Authors:
- Ren YangRYRen Yang
- Mai XuMXMai Xu
- Zulin WangZWZulin Wang
- Tianyi LiTLTianyi Li
Top Cited Papers
- 1 June 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6664-6673
Abstract
The past few years have witnessed great success in applying deep learning to enhance the quality of compressed image/video. The existing approaches mainly focus on enhancing the quality of a single frame, ignoring the similarity between consecutive frames. In this paper, we investigate that heavy quality fluctuation exists across compressed video frames, and thus low quality frames can be enhanced using the neighboring high quality frames, seen as Multi-Frame Quality Enhancement (MFQE). Accordingly, this paper proposes an MFQE approach for compressed video, as a first attempt in this direction. In our approach, we firstly develop a Support Vector Machine (SVM) based detector to locate Peak Quality Frames (PQFs) in compressed video. Then, a novel Multi-Frame Convolutional Neural Network (MF-CNN) is designed to enhance the quality of compressed video, in which the non-PQF and its nearest two PQFs are as the input. The MF-CNN compensates motion between the non-PQF and PQFs through the Motion Compensation subnet (MC-subnet). Subsequently, the Quality Enhancement subnet (QE-subnet) reduces compression artifacts of the non-PQF with the help of its nearest PQFs. Finally, the experiments validate the effectiveness and generality of our MFQE approach in advancing the state-of-the-art quality enhancement of compressed video. The code of our MFQE approach is available at https://github.com/ryangBUAA/MFQE.git.Keywords
All Related Versions
This publication has 28 references indexed in Scilit:
- Compression Artifacts Reduction by a Deep Convolutional Network2015
- Adaptive non-local means filter for image deblockingSignal Processing: Image Communication, 2013
- Image deblocking via sparse representationSignal Processing: Image Communication, 2012
- LIBSVMACM Transactions on Intelligent Systems and Technology, 2011
- Blocking Artifacts Suppression in Block-Coded Images Using Overcomplete Wavelet RepresentationIEEE Transactions on Circuits and Systems for Video Technology, 2004
- Low-delay rate control for DCT video coding via ρ-domain source modelingIEEE Transactions on Circuits and Systems for Video Technology, 2001
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- The MPEG-4 video standard verification modelIEEE Transactions on Circuits and Systems for Video Technology, 1997
- Digital video coding standards and their role in video communicationsProceedings of the IEEE, 1995
- The MPEG video compression algorithmSignal Processing: Image Communication, 1992