Depth Enhancement via Low-Rank Matrix Completion
Authors:
- Si LuSLSi Lu
- Xiaofeng RenXRXiaofeng Ren
- Feng LiuFLFeng Liu
- 1 June 2014
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636919,p. 3390-3397
Abstract
Depth captured by consumer RGB-D cameras is often noisy and misses values at some pixels, especially around object boundaries. Most existing methods complete the missing depth values guided by the corresponding color image. When the color image is noisy or the correlation between color and depth is weak, the depth map cannot be properly enhanced. In this paper, we present a depth map enhancement algorithm that performs depth map completion and de-noising simultaneously. Our method is based on the observation that similar RGB-D patches lie in a very low-dimensional subspace. We can then assemble the similar patches into a matrix and enforce this low-rank subspace constraint. This low-rank subspace constraint essentially captures the underlying structure in the RGB-D patches and enables robust depth enhancement against the noise or weak correlation between color and depth. Based on this subspace constraint, our method formulates depth map enhancement as a low-rank matrix completion problem. Since the rank of a matrix changes over matrices, we develop a data-driven method to automatically determine the rank number for each matrix. The experiments on both public benchmarks and our own captured RGB-D images show that our method can effectively enhance depth maps.Keywords
This publication has 24 references indexed in Scilit:
- Structure guided fusion for depth map inpaintingPattern Recognition Letters, 2013
- Coherent Spatiotemporal Filtering, Upsampling and Rendering of RGBZ VideosComputer Graphics Forum, 2012
- From learning models of natural image patches to whole image restoration2011
- High quality depth map upsampling for 3D-TOF cameras2011
- KinectFusion2011
- Temporal filtering for depth maps generated by Kinect depth camera2011
- Joint bilateral upsamplingACM Transactions on Graphics, 2007
- Learning Conditional Random Fields for Stereo2007
- Robust edge detection in noisy imagesComputational Statistics & Data Analysis, 2006
- Fields of Experts: A Framework for Learning Image Priors2005