PCSGAN: Perceptual Cyclic-Synthesized Generative Adversarial Networks for Thermal and NIR to Visible Image Transformation
Authors:
- Kancharagunta Kishan BabuKBKancharagunta Kishan Babu
- Shiv Ram DubeySDShiv Ram Dubey
Preprint
Abstract
In many real world scenarios, it is difficult to capture the images in the visible light spectrum (VIS) due to bad lighting conditions. However, the images can be captured in such scenarios using Near-Infrared (NIR) and Thermal (THM) cameras. The NIR and THM images contain the limited details. Thus, there is a need to transform the images from THM/NIR to VIS for better understanding. However, it is non-trivial task due to the large domain discrepancies and lack of abundant datasets. Nowadays, Generative Adversarial Network (GAN) is able to transform the images from one domain to another domain. Most of the available GAN based methods use the combination of the adversarial and the pixel-wise losses (like L1 or L2) as the objective function for training. The quality of transformed images in case of THM/NIR to VIS transformation is still not up to the mark using such objective function. Thus, better objective functions are needed to improve the quality, fine details and realism of the transformed images. A new model for THM/NIR to VIS image transformation called Perceptual Cyclic-Synthesized Generative Adversarial Network (PCSGAN) is introduced to address these issues. The PCSGAN uses the combination of the perceptual (i.e., feature based) losses along with the pixel-wise and the adversarial losses. Both the quantitative and qualitative measures are used to judge the performance of the PCSGAN model over the WHU-IIP face and the RGB-NIR scene datasets. The proposed PCSGAN outperforms the state-of-the-art image transformation models, including Pix2pix, DualGAN, CycleGAN, PS2GAN, and PAN in terms of the SSIM, MSE, PSNR and LPIPS evaluation measures. The code is available at: \url{https://github.com/KishanKancharagunta/PCSGAN}.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- CDGAN: Cyclic Discriminative Generative Adversarial Networks for image-to-image transformationJournal of Visual Communication and Image Representation, 2022
- Triple-translation GAN with multi-layer sparse representation for face image synthesisNeurocomputing, 2019
- Unpaired photo-to-caricature translation on faces in the wildNeurocomputing, 2019
- Matching Thermal to Visible Face Images Using a Semantic-Guided Generative Adversarial Network2019
- Multi-scale semantic image inpainting with residual learning and GANNeurocomputing, 2019
- Saliency detection via conditional adversarial image-to-image networkNeurocomputing, 2018
- Thermal to Visible Facial Image Translation Using Generative Adversarial NetworksIEEE Signal Processing Letters, 2018
- Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network2017
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004