Abstract
Due to occlusion among views and temporal inconsistency in depth video, spatio-temporal distortion occurs in 3D synthesized video with depth image-based rendering. In this paper, we propose a deep Convolutional Neural Network (CNN)-based synthesized video denoising algorithm to reduce temporal flicker distortion and improve perceptual quality of 3D synthesized video. First, we analyze the spatio-temporal distortion, and model eliminating spatio-temporal distortion as a perceptual video denoising problem. Then, a deep learning-based synthesized video denoising network is proposed, in which a CNN-friendly spatio-temporal loss function is derived from a synthesized video quality metric and integrated with a single image denoising network architecture. Finally, specific schemes, i.e., specific Synthesized Video Denoising Networks (SynVD-Nets), and a general scheme, i.e., General SynVD-Net (GSynVD-Net), based on existing CNN-based denoising models, are developed to handle synthesized video with different distortion levels more effectively. Experimental results show that the proposed SynVD-Net and GSynVD-Net can outperform deep learning-based counterparts and conventional denoising methods, and significantly enhance perceptual quality of 3D synthesized video.
Funding Information
  • National Natural Science Foundation of China (62172400, 61901459)
  • Shenzhen Science and Technology Program (JCYJ20180507183823045, JCYJ20200109110410133)
  • Guangdong International Science and Technology Cooperative Research Project (2018A050506063)
  • Membership of Youth Innovation Promotion Association, Chinese Academy of Sciences (2018392)
  • China Postdoctoral Science Foundation (2021T140696)

This publication has 46 references indexed in Scilit: