Latent Diffusion, Implicit Amplification: Efficient Continuous-Scale Super-Resolution for Remote Sensing Images
Authors:
- Hanlin WuHWHanlin Wu
- Jiangwei MoJMJiangwei Mo
- Xiaohui SunXSXiaohui Sun
- Jie MaJMJie Ma
Preprint
Abstract
Recent advancements in diffusion models have significantly improved performance in super-resolution (SR) tasks. However, previous research often overlooks the fundamental differences between SR and general image generation. General image generation involves creating images from scratch, while SR focuses specifically on enhancing existing low-resolution (LR) images by adding typically missing high-frequency details. This oversight not only increases the training difficulty but also limits their inference efficiency. Furthermore, previous diffusion-based SR methods are typically trained and inferred at fixed integer scale factors, lacking flexibility to meet the needs of up-sampling with non-integer scale factors. To address these issues, this paper proposes an efficient and elastic diffusion-based SR model (E$^2$DiffSR), specially designed for continuous-scale SR in remote sensing imagery. E$^2$DiffSR employs a two-stage latent diffusion paradigm. During the first stage, an autoencoder is trained to capture the differential priors between high-resolution (HR) and LR images. The encoder intentionally ignores the existing LR content to alleviate the encoding burden, while the decoder introduces an SR branch equipped with a continuous scale upsampling module to accomplish the reconstruction under the guidance of the differential prior. In the second stage, a conditional diffusion model is learned within the latent space to predict the true differential prior encoding. Experimental results demonstrate that E$^2$DiffSR achieves superior objective metrics and visual quality compared to the state-of-the-art SR methods. Additionally, it reduces the inference time of diffusion-based SR methods to a level comparable to that of non-diffusion methods.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- MSAGAN: A New Super-Resolution Algorithm for Multispectral Remote Sensing Image Based on a Multiscale Attention GAN NetworkIEEE Geoscience and Remote Sensing Letters, 2023
- Learning Dynamic Scale Awareness and Global Implicit Functions for Continuous-Scale Super-Resolution of Remote Sensing ImagesIEEE Transactions on Geoscience and Remote Sensing, 2023
- SRDiff: Single image super-resolution with diffusion probabilistic modelsNeurocomputing, 2022
- Lightweight Image Super-Resolution With Expectation-Maximization Attention MechanismIEEE Transactions on Circuits and Systems for Video Technology, 2021
- Remote Sensing Image Super-Resolution Using Novel Dense-Sampling NetworksIEEE Transactions on Geoscience and Remote Sensing, 2020
- Object detection in optical remote sensing images: A survey and a new benchmarkISPRS Journal of Photogrammetry and Remote Sensing, 2020
- Coupled Adversarial Training for Remote Sensing Image Super-ResolutionIEEE Transactions on Geoscience and Remote Sensing, 2019
- Super-Resolution of Single Remote Sensing Image Based on Residual Dense Backprojection NetworksIEEE Transactions on Geoscience and Remote Sensing, 2019
- Super-resolution of Sentinel-2 images: Learning a globally applicable deep neural networkISPRS Journal of Photogrammetry and Remote Sensing, 2018
- AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene ClassificationIEEE Transactions on Geoscience and Remote Sensing, 2017