Joint Camera Spectral Response Selection and Hyperspectral Image Recovery
- 17 July 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 44 (1) , 256-272
Abstract
Hyperspectral image (HSI) recovery from a single RGB image has attracted much attention, whose performance has recently been shown to be sensitive to the camera spectral response (CSR). In this paper, we present an efficient convolutional neural network (CNN) based method, which can jointly select the optimal CSR from a candidate dataset and learn a mapping to recover HSI from a single RGB image captured with this algorithmically selected camera under multi-chip or single-chip setups. Given a specific CSR, we first present a HSI recovery network, which accounts for the underlying characteristics of the HSI, including spectral nonlinear mapping and spatial similarity. Later, we append a CSR selection layer onto the recovery network, and the optimal CSR under both multi-chip and single-chip setups can be automatically determined from the network weights under nonnegative sparse constraint. Experimental results on three hyperspectral datasets and two camera spectral response datasets demonstrate that our HSI recovery network outperforms state-of-the-art methods in terms of both quantitative metrics and perceptive quality, and the selection layer always returns a CSR consistent to the best one determined by exhaustive search. Finally, we collect a hyperspectral flower dataset to evaluate the effect from HSI recovery on classification problem.Keywords
Funding Information
- National Natural Science Foundation of China (61672096)
- JSPS KAKENHI (19K20307)
This publication has 64 references indexed in Scilit:
- Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: A review — Part I: FundamentalsInnovative Food Science & Emerging Technologies, 2013
- Camera Spectral Sensitivity and White Balance Estimation from Sky ImagesInternational Journal of Computer Vision, 2013
- Visual enhancement of old documents with hyperspectral imagingPattern Recognition, 2011
- Segmentation and classification of hyperspectral images using watershed transformationPattern Recognition, 2010
- Snapshot Image Mapping Spectrometer (IMS) with high sampling density for hyperspectral microscopyOptics Express, 2010
- Multi-Spectral Imaging by Optimized Wide Band IlluminationInternational Journal of Computer Vision, 2008
- Single-shot compressive spectral imaging with a dual-disperser architectureOptics Express, 2007
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- Large-image-format computed tomography imaging spectrometer for fluorescence microscopyOptics Express, 2001
- The spectral image processing system (SIPS)—interactive visualization and analysis of imaging spectrometer dataRemote Sensing of Environment, 1993