DCPNet: A Dual-Task Collaborative Promotion Network for Pansharpening
- 18 March 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 62 (01962892) , 1-16
Abstract
Pansharpening, a type of image fusion, combines a high-resolution panchromatic (PAN) image with a low-resolution multispectral (LRMS) image to produce a high-resolution multispectral output. Recent advancements in deep learning have led to the development of data-driven pansharpening methods, which have shown superior performance over traditional non-data-driven approaches. Nevertheless, current data-driven methods primarily focus on the design of single-task-driven networks. They tend to ignore the positive impact that incorporating auxiliary tasks could have on pansharpening. This oversight restricts further improvements in their performance. To break the constraints of single-task methods, we propose a dual-task collaborative promotion network (DCPNet) for pansharpening. DCPNet incorporates an LRMS image super-resolution (SR) reconstruction network into the pansharpening network, establishing a dual-task parallel collaborative framework that achieves joint collaborative optimization for both pansharpening and SR reconstruction. Additionally, we devise a feature fusion scheme that jointly reinforces spectral and spatial details, establishing a bridge for information interaction between the two task branches. This enables the two branches to extract higher quality features in a collaborative way, thus facilitating the realization of the interaction and fusion of dual-task features, as well as the exploration of potential information within the fused features. Experimental results demonstrate that our approach not only effectively transfers spectral information from the LRMS images to the fusion result but also preserves spatial details from the PAN images. The code is available at https://github.com/lhf12278/DCPNet.Keywords
Funding Information
- Yunnan Fundamental Research Projects (202301AV070004)
- National Natural Science Foundation of China (62161015)
This publication has 53 references indexed in Scilit:
- Cross-city matters: A multimodal remote sensing benchmark dataset for cross-city semantic segmentation using high-resolution domain adaptation networksRemote Sensing of Environment, 2023
- UPanGAN: Unsupervised pansharpening based on the spectral and spatial loss constrained Generative Adversarial NetworkInformation Fusion, 2022
- Fast and accurate multi-class geospatial object detection with large-size remote sensing imagery using CNN and Truncated NMSISPRS Journal of Photogrammetry and Remote Sensing, 2022
- Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection2022
- Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion networkInformation Fusion, 2022
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows2021
- Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusionInformation Fusion, 2020
- Monitoring tropical forest degradation using spectral unmixing and Landsat time series analysisRemote Sensing of Environment, 2020
- Squeeze-and-Excitation Networks2018
- PanNet: A Deep Network Architecture for Pan-Sharpening2017