Learning joint intensity-depth sparse representations
Preprint
Abstract
This paper presents a method for learning overcomplete dictionaries composed of two modalities that describe a 3D scene: image intensity and scene depth. We propose a novel Joint Basis Pursuit (JBP) algorithm that finds related sparse features in two modalities using conic programming and integrate it into a two-step dictionary learning algorithm. JBP differs from related convex algorithms because it finds joint sparsity models with different atoms and different coefficient values for intensity and depth. This is crucial for recovering generative models where the same sparse underlying causes (3D features) give rise to different signals (intensity and depth). We give a theoretical bound for the sparse coefficient recovery error obtained by JBP, and show experimentally that JBP is far superior to the state of the art Group Lasso algorithm. When applied to the Middlebury depth-intensity database, our learning algorithm converges to a set of related features, such as pairs of depth and intensity edges or image textures and depth slants. Finally, we show that the learned dictionary and JBP achieve the state of the art depth inpainting performance on time-of-flight 3D data.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Learning Sparse Representations of DepthIEEE Journal of Selected Topics in Signal Processing, 2011
- Multiview Imaging and 3DTVIEEE Signal Processing Magazine, 2007
- Image Denoising Via Sparse and Redundant Representations Over Learned DictionariesIEEE Transactions on Image Processing, 2006
- For most large underdetermined systems of equations, the minimal 𝓁1‐norm near‐solution approximates the sparsest near‐solutionCommunications on Pure and Applied Mathematics, 2006
- Algorithms for simultaneous sparse approximation. Part II: Convex relaxationSignal Processing, 2006
- Stable signal recovery from incomplete and inaccurate measurementsCommunications on Pure and Applied Mathematics, 2006
- An Algorithm for Total Variation Minimization and ApplicationsJournal of Mathematical Imaging and Vision, 2004
- A taxonomy and evaluation of dense two-frame stereo correspondence algorithms2002
- A convergence analysis of the scaling-invariant primal-dual path-following algorithms for second-order cone programmingOptimization Methods and Software, 1999
- Sparse coding with an overcomplete basis set: A strategy employed by V1?Vision Research, 1997