Abstract
Hyperspectral image fusion (HIF) is pivotal in scientific research and engineering. With the fast development of deep learning techniques, numerous HIF algorithms have been proposed. However, traditional HIF algorithms, which are designed specifically for different input conditions such as hyperspectral-multispectral image fusion (HMF), hyperspectral-panchromatic image fusion (HPF), and hyperspectral-multispectral-panchromatic image fusion (HMPF), cause significant inconvenience in the practical application of HIF. To overcome this issue, we propose a unified network (UniNet) that integrates HMF, HPF, and HMPF in an “all in one” manner. During testing, UniNet can perform HIF under different input conditions, enhancing its practicality. Specifically, the UniNet consists of the HMF, the HPF, and the HMPF sub-networks. Each sub-network is designed to reconstruct high-resolution hyperspectral image (HSI) under a specific input condition. A mutual learning strategy is proposed to establish the relationships among sub-networks, enabling the transfer of spectral-spatial knowledge across different input conditions. Additionally, a multi-scale gradient spatial attention module is designed in the network to better extract the spatial edge details of HSIs at multiple scales. Once trained, UniNet is capable of working with its sub-networks independently. Extensive experiments show that UniNet outperforms 18 state-of-the-art algorithms with five HSI datasets. Furthermore, the effectiveness of our algorithm is validated through ablation experiments.
Funding Information
  • National Key Research and Development Program of China (2022YFD1700802)
  • Key Research and Development Program of Hunan Province, China (2023SK2041)
  • Natural Science Foundation of Hainan Province (2023JJ20018)
  • Changsha Science and Technology Program (kq2208036)

This publication has 59 references indexed in Scilit: