Deep Light Field Acquisition Using Learned Coded Mask Distributions for Color Filter Array Sensors
- 5 May 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Imaging
- Vol. 7 (25730436) , 475-488
Abstract
Compressive light field photography enables light field acquisition using a single sensor by utilizing a color coded mask. This approach is very cost effective since consumer-level digital cameras can be turned into light field cameras by simply placing a coded mask between the sensor and the aperture plane. This paper describes a deep learning architecture for compressive light field acquisition using a color coded mask and a sensor with Color Filter Array (CFA). Unlike previous methods where a fixed mask pattern is used, our deep network learns the optimal distribution of the color coded mask pixels. The proposed solution enables end-to-end learning of the color-coded mask distribution and the reconstruction network, taking into account the sensor CFA. Consequently, the resulting network can efficiently perform joint demosaicing and light field reconstruction of images acquired with color-coded mask and a CFA sensor. Compared to previous methods based on deep learning with monochrome sensors, as well as traditional compressive sensing approaches using CFA sensors, we obtain superior color reconstruction of the light fields.Keywords
Funding Information
- EU H2020 Research and Innovation Programme (694122)
- ERC
- CLIM
- ANR research agency
- DeepCIM
This publication has 37 references indexed in Scilit:
- Compressive Image Reconstruction in Reduced Union of SubspacesComputer Graphics Forum, 2015
- An Improved RIP-Based Performance Guarantee for Sparse Signal Recovery via Orthogonal Matching PursuitIEEE Transactions on Information Theory, 2014
- Compressive light field photography using overcomplete dictionaries and optimized projectionsACM Transactions on Graphics, 2013
- A high-resolution lightfield camera with dual-mask design2012
- Programmable aperture photographyACM Transactions on Graphics, 2008
- Dappled photographyACM Transactions on Graphics, 2007
- Stable signal recovery from incomplete and inaccurate measurementsCommunications on Pure and Applied Mathematics, 2006
- High performance imaging using large camera arraysACM Transactions on Graphics, 2005
- Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition2002
- Light field rendering1996