Continuous Satellite Image Generation from Standard Layer Maps Using Conditional Generative Adversarial Networks
Open Access
- 11 December 2024
- journal article
- research article
- Published by MDPI AG in ISPRS International Journal of Geo-Information
- Vol. 13 (12) , 448
Abstract
Satellite image generation has a wide range of applications. For example, parts of images must be restored in areas obscured by clouds or cloud shadows or areas that must be anonymized. The need to cover a large area with the generated images faces the challenge that separately generated images must maintain the structural and color continuity between the adjacent generated images as well as the actual ones. This study presents a modified architecture of the generative adversarial network (GAN) pix2pix that ensures the integrity of the generated remote sensing images. The pix2pix model comprises a U-Net generator and a PatchGAN discriminator. The generator was modified by expanding the input set with images representing the known parts of ground truth and the respective mask. Data used for the generative model consist of Sentinel-2 (S2) RGB satellite imagery as the target data and OpenStreetMap mapping data as the input. Since forested areas and fields dominate in images, a Kneedle clusterization method was applied to create datasets that better represent the other classes, such as buildings and roads. The original and updated models were trained on different datasets and their results were evaluated using gradient magnitude (GM), Fréchet inception distance (FID), structural similarity index measure (SSIM), and multiscale structural similarity index measure (MS-SSIM) metrics. The models with the updated architecture show improvement in gradient magnitude, SSIM, and MS-SSIM values for all datasets. The average GMs of the junction region and the full image are similar (do not exceed 7%) for the images generated using the modified architecture whereas it is more than 13% higher in the junction area for the images generated using the original architecture. The importance of class balancing is demonstrated by the fact that, for both architectures, models trained on the dataset with a higher ratio of classes representing buildings and roads compared to the models trained on the dataset without clusterization have more than 10% lower FID (162.673 to 190.036 for pix2pix and 173.408 to 195.621 for the modified architecture) and more than 5% higher SSIM (0.3532 to 0.3284 for pix2pix and 0.3575 to 0.3345 for the modified architecture) and MS-SSIM (0.3532 to 0.3284 for pix2pix and 0.3575 to 0.3345 for the modified architecture) values.Keywords
Funding Information
- European Commission (101059903)
This publication has 31 references indexed in Scilit:
- CERMF-Net: A SAR-Optical Feature Fusion for Cloud Elimination From Sentinel-2 Imagery Using Residual Multiscale Dilated NetworkIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2024
- Damage GAN: A Generative Model for Imbalanced Data2023
- PPUP-GAN: A GAN-based privacy-protecting method for aerial photographyFuture Generation Computer Systems, 2023
- Uncertainty-Aware Semantic Guidance and Estimation for Image InpaintingIEEE Journal of Selected Topics in Signal Processing, 2020
- Single Image Cloud Removal Using U-Net and Generative Adversarial NetworksIEEE Transactions on Geoscience and Remote Sensing, 2020
- Monitoring agriculture areas with satellite images and deep learningApplied Soft Computing, 2020
- Remote sensing for agricultural applications: A meta-reviewRemote Sensing of Environment, 2020
- Assessing global Sentinel-2 coverage dynamics and data availability for operational Earth observation (EO) applications using the EO-CompassInternational Journal of Digital Earth, 2019
- Global characterization and monitoring of forest cover using Landsat data: opportunities and challengesInternational Journal of Digital Earth, 2012
- Multiscale structural similarity for image quality assessment2004