AMPUNet: Hierarchical Attention Map Pyramid for Semantic Segmentation of Remote Sensing Images
Open Access
- 20 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 19 (19391404) , 6328-6340
Abstract
Accurate semantic segmentation of high-resolution remote sensing imagery is essential for applications ranging from urban planning to environmental monitoring. However, this task remains fundamentally challenging due to the complex spatial patterns, extreme scale variations, and fine-grained details inherent in geographical scenes. While attention mechanisms, particularly global and sparse attention, have shown promise in capturing long-range dependencies, existing approaches often suffer from three interconnected limitations: prohibitive computational complexity, misalignment when integrating multi-scale representations, and loss of semantic information during the decoder's upsampling stages. This paper introduces AMPUNet, a novel framework designed to overcome these limitations through the construction of a hierarchical, coarse-to-fine attention map pyramid. Our core innovation lies in explicitly propagating and refining attention maps across network layers rather than operating solely on feature maps. Specifically, we design: (1) a hybrid sparse attention framework combining a Block Attention Module and a Column Attention Module to model global context efficiently; (2) a Dimension Correspondence Module to achieve tensor-level granularity alignment for multi-scale attention maps; and (3) an Attention Map Merging Module with a Cross-layer Attention Weighting strategy, which directly transfers high-level semantic information from deep to shallow layers, mitigating information degradation. Extensive experiments on the ISPRS Vaihingen, Potsdam, and LoveDA benchmarks demonstrate that AMPUNet achieves superior performance, with mIoU scores of 75.43% on Vaihingen, 78.03% on Potsdam, and 50.94% on LoveDA, while maintaining competitive inference efficiency. Our findings confirm that structuring attention into a learnable pyramid is a highly effective paradigm for remote sensing semantic segmentation, successfully balancing precise detail preservation with robust global understanding. The source code is available at: https://github.com/interpretty/AMPUNet.Keywords
Funding Information
- National Key Research and Development Program of China (2022YFF0904400)
This publication has 71 references indexed in Scilit:
- CT-HiffNet: A contour-texture hierarchical feature fusion network for cropland field parcel extraction from high-resolution remote sensing imagesComputers and Electronics in Agriculture, 2025
- Enhanced multi-scale feature adaptive fusion sparse convolutional network for large-scale scenes semantic segmentationComputers & Graphics, 2024
- A Novel Remote Sensing Landslide Semantic Segmentation Method: Using CycleGAN-Based Change Detection Algorithms2024
- Hybrid Shunted Transformer embedding UNet for remote sensing image semantic segmentationNeural Computing and Applications, 2024
- Dialogue emotion model based on local–global context encoder and commonsense knowledge fusion attentionInternational Journal of Machine Learning and Cybernetics, 2024
- Statistical texture involved multi-granularity attention network for remote sensing semantic segmentationMultimedia Tools and Applications, 2023
- Optimizing block morphology for reducing traffic pollutant concentration in adjacent external spaces of street canyons: A machine learning approachBuilding and Environment, 2023
- Attention mechanisms in computer vision: A surveyComputational Visual Media, 2022
- A hybrid-attention semantic segmentation network for remote sensing interpretation in land-use surveillanceInternational Journal of Machine Learning and Cybernetics, 2022
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015