Deep Color Guided Coarse-to-Fine Convolutional Network Cascade for Depth Image Super-Resolution
- 8 October 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 28 (2) , 994-1006
Abstract
Depth image super-resolution is a significant yet challenging task. In this paper, we introduce a novel deep color guided coarse-to-fine convolutional neural network (CNN) framework to address this problem. First, we present a data-driven filter method to approximate the ideal filter for depth image super-resolution instead of hand-designed filters. Based on large data samples, the filter learned is more accurate and stable for upsampling depth image. Second, we introduce a coarse-to-fine CNN to learn different sizes of filter kernels. In the coarse stage, larger filter kernels are learned by the CNN to achieve crude high-resolution depth image. As to the fine stage, the crude high-resolution depth image is used as the input so that smaller filter kernels are learned to gain more accurate results. Benefit from this network, we can progressively recover the high frequency details. Third, we construct a color guidance strategy that fuses color difference and spatial distance for depth image upsampling. We revise the interpolated high-resolution depth image according to the corresponding pixels in high-resolution color maps. Guided by color information, the depth of high-resolution image obtained can alleviate texture copying artifacts and preserve edge details effectively. Quantitative and qualitative experimental results demonstrate our state-of-the-art performance for depth map super-resolution.Keywords
Funding Information
- National Natural Science Foundation of China (61872241, 61572316, 61671290, 61471235)
- National Key Research and Development Program of China (2016YFC1300302, 2017YFE0104000)
- Science and Technology Commission of Shanghai Municipality (16DZ0501100, 17411952600)
- Fundo para o Desenvolvimento das Ciências e da Tecnologia (0027/2018/A1)
- Shanghai “The Belt and Road” Young Scholar Exchange Grant (17510740100)
This publication has 50 references indexed in Scilit:
- Depth map super-resolution using stereo-vision-assisted modelNeurocomputing, 2015
- High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth2014
- Stereo Matching Using Tree FilteringIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Color-Guided Depth Recovery From RGB-D Data Using an Adaptive Autoregressive ModelIEEE Transactions on Image Processing, 2014
- Object Movements Synopsis viaPart Assembling and StitchingIEEE Transactions on Visualization and Computer Graphics, 2014
- Real-time human pose recognition in parts from single depth imagesCommunications of the ACM, 2013
- On Single Image Scale-Up Using Sparse-Representations2012
- Towards Semantic Scene Analysis with Time-of-Flight Cameras2011
- Joint bilateral upsamplingACM Transactions on Graphics, 2007
- Segmentation-Based Adaptive Support for Accurate Stereo Correspondence2007