Efficient Multi‐image Correspondences for On‐line Light Field Video Processing
- 27 October 2016
- journal article
- Published by Wiley in Computer Graphics Forum
- Vol. 35 (7) , 401-410
Abstract
Light field videos express the entire visual information of an animated scene, but their shear size typically makes capture, processing and display anoff‐lineprocess, i. e., time between initial capture and final display is far from real‐time. In this paper we propose a solution for one of the key bottlenecks in such a processing pipeline, which is a reliable depth reconstruction possibly for many views. This is enabled by a novel correspondence algorithm converting the video streams from a sparse array of off‐the‐shelf cameras into an array of animated depth maps. The algorithm is based on a generalization of the classic multi‐resolution Lucas‐Kanade correspondence algorithm from a pair of images to an entire array. Special inter‐image confidence consolidation allows recovery from unreliable matching in some locations and some views. It can be implemented efficiently in massively parallel hardware, allowing for interactive computations. The resulting depth quality as well as the computation performance compares favorably to other state‐of‐the art light field‐to‐depth approaches, as well as stereo matching techniques. Another outcome of this work is a data set of light field videos that are captured with multiple variants of sparse camera arrays.Keywords
This publication has 31 references indexed in Scilit:
- Camera array calibration for light field acquisitionFrontiers of Computer Science, 2015
- PiCamACM Transactions on Graphics, 2013
- Variational Light Field Analysis for Disparity Estimation and Super-ResolutionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Single lens 3D-camera with extended depth-of-field2012
- Stereo Processing by Semiglobal Matching and Mutual InformationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2007
- Joint bilateral upsamplingACM Transactions on Graphics, 2007
- High performance imaging using large camera arraysACM Transactions on Graphics, 2005
- Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow MethodsInternational Journal of Computer Vision, 2005
- Real‐Time Consensus‐Based Scene Reconstruction Using Commodity Graphics Hardware†Computer Graphics Forum, 2003
- The Laplacian Pyramid as a Compact Image CodeIEEE Transactions on Communications, 1983