An Integrated Approach to Registration and Fusion of Hyperspectral and Multispectral Images
- 12 November 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 58 (5) , 3020-3033
Abstract
Combining a hyperspectral (HS) image and a multispectral (MS) image—an example of image fusion—can result in a spatially and spectrally high-resolution image. Despite the plethora of fusion algorithms in remote sensing, a necessary prerequisite, namely registration, is mostly ignored. This limits their application to well-registered images from the same source. In this article, we propose and validate an integrated registration and fusion approach (code available at https://github.com/zhouyuanzxcv/Hyperspectral ). The registration algorithm minimizes a least-squares (LSQ) objective function with the point spread function (PSF) incorporated together with a nonrigid freeform transformation applied to the HS image and a rigid transformation applied to the MS image. It can handle images with significant scale differences and spatial distortion. The fusion algorithm takes the full high-resolution HS image as an unknown in the objective function. Assuming that the pixels lie on a low-dimensional manifold invariant to local linear transformations from spectral degradation, the fusion optimization problem leads to a closed-form solution. The method was validated on the Pavia University, Salton Sea, and the Mississippi Gulfport datasets. When the proposed registration algorithm is compared to its rigid variant and two mutual information-based methods, it has the best accuracy for both the nonrigid simulated dataset and the real dataset, with an average error less than 0.15 pixels for nonrigid distortion of maximum 1 HS pixel. When the fusion algorithm is compared with current state-of-the-art algorithms, it has the best performance on images with registration errors as well as on simulations that do not consider registration effects.Keywords
Funding Information
- NSF IIS (NSF IIS 1743050)
This publication has 44 references indexed in Scilit:
- Multi- and hyperspectral geologic remote sensing: A reviewInternational Journal of Applied Earth Observation and Geoinformation, 2012
- MAP Estimation for Hyperspectral Image Resolution Enhancement Using an Auxiliary SensorIEEE Transactions on Image Processing, 2004
- A new point matching algorithm for non-rigid registrationComputer Vision and Image Understanding, 2003
- Extension of phase correlation to subpixel registrationIEEE Transactions on Image Processing, 2002
- Impact of sensor's point spread function on land cover characterization: assessment and deconvolutionRemote Sensing of Environment, 2002
- Fully constrained least squares linear spectral mixture analysis method for material quantification in hyperspectral imageryIEEE Transactions on Geoscience and Remote Sensing, 2001
- Nonlinear Dimensionality Reduction by Locally Linear EmbeddingScience, 2000
- Unmixing-based multisensor multiresolution image fusionIEEE Transactions on Geoscience and Remote Sensing, 1999
- Mapping Chaparral in the Santa Monica Mountains Using Multiple Endmember Spectral Mixture ModelsRemote Sensing of Environment, 1998
- An FFT-based technique for translation, rotation, and scale-invariant image registrationIEEE Transactions on Image Processing, 1996