Abstract
This paper proposes a new generative adversarial network for infrared and visible image fusion based on semantic segmentation (SSGAN), which can consider not only the low-level features of infrared and visible images, but also the high-level semantic information. Source images can be divided into foregrounds and backgrounds by semantic masks. The generator with a dual-encoder-single-decoder framework is used to extract the feature of foregrounds and backgrounds by different encoder paths. Moreover, the discriminator’s input image is designed based on semantic segmentation, which is obtained by combining the foregrounds of the infrared images with the backgrounds of the visible images. Consequently, the prominence of thermal targets in the infrared images and texture details in the visible images can be preserved in the fused images simultaneously. Qualitative and quantitative experiments on publicly available datasets demonstrate that the proposed approach can significantly outperform the state-of-the-art methods.
Funding Information
  • National Natural Science Foundation of China (61771353, 61773295, 62072350, 41501505)
  • Hubei Technology Innovation Project (2019AAA045)
  • Innovation Fund for Industry, Education and Research in Chinese Universities-New Generation Information Technology Innovation Project (2019ITA02014)
  • the Key Scientific and Technological Research Project of Hubei Provincial Education Department (D20201507)

This publication has 50 references indexed in Scilit: