Abstract
The quality of synthesized view plays an important role in the 3D video system. In this paper, to further improve the coding efficiency, a convolutional neural network (CNN)-based synthesized view quality enhancement method for 3D high efficiency video coding (HEVC) is proposed. First, the distortion elimination in synthesized view is formulated as an image restoration task with the aim to reconstruct the latent distortion free synthesized image. Second, the learned CNN models are incorporated into 3D HEVC codec to improve the view synthesis performance for both view synthesis optimization (VSO) and the final synthesized view, where the geometric and compression distortions are considered according to the specific characteristics of synthesized view. Third, a new Lagrange multiplier in the rate-distortion cost function is derived to adapt the CNN-based VSO process to embrace a better 3D video coding performance. Extensive experimental results show that the proposed scheme can efficiently eliminate the artifacts in the synthesized image, and reduce 25.9% and 11.7% bit rate in terms of peak-signal-to-noise ratio and structural similarity index, which significantly outperforms the state-of-the-art methods.
Funding Information
  • National Natural Science Foundation of China (61672443, 61471348, 61571274)
  • Hong Kong RGC General Research Fund (9042322 (CityU 11200116), 9042489 (CityU 11206317))
  • Hong Kong RGC Early Career Scheme (9048122 (CityU 21211018))
  • City University of Hong Kong (7200539/CS)
  • Guangdong Natural Science Funds for Distinguished Young Scholar (2016A030306022)
  • Project for Shenzhen Science and Technology Development (JSGG20160229202345378)
  • Shenzhen International Collaborative Research Project (GJHZ20170314155404913)
  • Guangdong Provincial Department of Science and Technology (2017B010110014)
  • Guangdong Special Support Program for Youth Science and Technology Innovation Talents (2014TQ01X345)
  • Youth Innovation Promotion Association of the Chinese Academy of Sciences (2018392)
  • Shandong Natural Science Funds for Distinguished Young Scholar (JQ201614)

This publication has 32 references indexed in Scilit: