Mingsong Li
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Mingsong Li has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Gongping Yang
Shandong University
10 shared publications - Yikun Liu
Shandong Management University
9 shared publications - Yuwen Huang
Heze University
6 shared publications - Guangkuo Xue
School of Software, Shandong University, Jinan, China
3 shared publications - Ruifan Zhang
State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering, Beihang University, Beijing, China
2 shared publications - Wenkai Yan
School of Software, Shandong University, Jinan, China
2 shared publications
Latest Publications
- Open Access
- 4 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 18 (19391404) , 160-175
Laplacian Pyramid Network With Hybrid Encoder and Edge Guidance for Remote Sensing Change Detection
Remote sensing change detection (CD) is a crucial task for observing and analyzing dynamic land cover alterations. Many CD methods based on deep learning demonstrate strong performance, but their effectiveness is influenced by the choice of encoder and the challenge of accurately delineating the edges of change regions. In this article, we propose a Laplacian pyramid network with hybrid encoder and edge guidance (HE-LPNet) to solve these issues. Specifically, the hybrid encoder combines the advantages of convolutional neural networks and transformer, resulting in extracted features that are more fine-grained. Meanwhile, the hybrid encoder incorporates the vision foundation models, leading to enhanced generalization of the overall model. In addition to feature extraction, the image is processed to generate a Laplacian pyramid, which is then fused with the features extracted by the hybrid encoder to enhance the salient features at the pixel-level. In the decoder stage, weighted guided attention is designed to selectively apply channel and spatial attention to the fused features, improving the network's ability to discriminate change regions. Furthermore, we present an edge-guided loss to capture edge information in change regions. To validate the effectiveness of the proposed HE-LPNet, extensive experiments are conducted on three high-resolution remote sensing CD datasets. The experimental results demonstrate that our method surpasses other state-of-the-art CD methods.
- 4 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 18 (19391404) , 160-175
Open AccessLaplacian Pyramid Network With Hybrid Encoder and Edge Guidance for Remote Sensing Change Detection
Remote sensing change detection (CD) is a crucial task for observing and analyzing dynamic land cover alterations. Many CD methods based on deep learning demonstrate strong performance, but their effectiveness is influenced by the choice of encoder and the challenge of accurately delineating the edges of change regions. In this article, we propose a Laplacian pyramid network with hybrid encoder and edge guidance (HE-LPNet) to solve these issues. Specifically, the hybrid encoder combines the advantages of convolutional neural networks and transformer, resulting in extracted features that are more fine-grained. Meanwhile, the hybrid encoder incorporates the vision foundation models, leading to enhanced generalization of the overall model. In addition to feature extraction, the image is processed to generate a Laplacian pyramid, which is then fused with the features extracted by the hybrid encoder to enhance the salient features at the pixel-level. In the decoder stage, weighted guided attention is designed to selectively apply channel and spatial attention to the fused features, improving the network's ability to discriminate change regions. Furthermore, we present an edge-guided loss to capture edge information in change regions. To validate the effectiveness of the proposed HE-LPNet, extensive experiments are conducted on three high-resolution remote sensing CD datasets. The experimental results demonstrate that our method surpasses other state-of-the-art CD methods.
- 22 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 62 (01962892) , 1-14
A Position-Temporal Awareness Transformer for Remote Sensing Change Detection
With the development of deep learning, significant progress has been made in change detection (CD) methods for remote sensing (RS) images. However, many convolutional neural network (CNN)-based methods are constrained in capturing long-range dependencies due to the limitations of the receptive field. Transformers rely on self-attention mechanisms to effectively achieve global information modeling and are widely used in CD tasks. Nevertheless, transformer-based CD methods still suffer from issues such as pseudochanges and incomplete edges due to the lack of position and temporal correlations in bitemporal RS images. To deal with this issue, we propose a position-temporal awareness transformer (PT-Former), which models position and temporal relations in bitemporal images. Specifically, a Siamese network attached to a position-aware embedding module (PEM) serves as a feature encoder to extract the features of changed areas. Then, a temporal difference perception module (TDPM) is designed to capture the cross-temporal shift and enhance the difference perception ability during cross-temporal interaction. Meanwhile, the contextual information of the ground object is aggregated by the fusion block, and the spatial relation is reconstructed under the guidance of bitemporal features. The experimental results validate the superiority of PT-Former on three benchmark datasets, including the season-varying CD (SVCD) dataset, the learning vision and RS laboratory building CD (LEVIR-CD) dataset, and the WHU-CD dataset confirming the potential of PT-Former for CD tasks in RS images. The code will be available at https://github.com/liuyk29/PT-Former .
- 22 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 62 (01962892) , 1-14
A Position-Temporal Awareness Transformer for Remote Sensing Change Detection
With the development of deep learning, significant progress has been made in change detection (CD) methods for remote sensing (RS) images. However, many convolutional neural network (CNN)-based methods are constrained in capturing long-range dependencies due to the limitations of the receptive field. Transformers rely on self-attention mechanisms to effectively achieve global information modeling and are widely used in CD tasks. Nevertheless, transformer-based CD methods still suffer from issues such as pseudochanges and incomplete edges due to the lack of position and temporal correlations in bitemporal RS images. To deal with this issue, we propose a position-temporal awareness transformer (PT-Former), which models position and temporal relations in bitemporal images. Specifically, a Siamese network attached to a position-aware embedding module (PEM) serves as a feature encoder to extract the features of changed areas. Then, a temporal difference perception module (TDPM) is designed to capture the cross-temporal shift and enhance the difference perception ability during cross-temporal interaction. Meanwhile, the contextual information of the ground object is aggregated by the fusion block, and the spatial relation is reconstructed under the guidance of bitemporal features. The experimental results validate the superiority of PT-Former on three benchmark datasets, including the season-varying CD (SVCD) dataset, the learning vision and RS laboratory building CD (LEVIR-CD) dataset, and the WHU-CD dataset confirming the potential of PT-Former for CD tasks in RS images. The code will be available at https://github.com/liuyk29/PT-Former .
- Open Access
- 13 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 17 (19391404) , 11761-11776
Exploring the Cross-Temporal Interaction: Feature Exchange and Enhancement for Remote Sensing Change Detection
Change detection (CD) aims to identify surface changes from bi-temporal remote sensing (RS) images, which is a crucial and challenging topic in RS. In recent years, RS images CD has achieved significant advancements through the use of convolutional neural networks (CNNs). However, existing deep learning-based CD methods still face some challenges such as blurry boundaries, pseudo changes caused by fluctuations in imaging conditions, and complexity of change objects in the scene. In this study, we present a novel perspective for exploring the cross-temporal interaction in the feature fusion stage and propose a bi-temporal feature exchange and enhancement network (ExNet). Specifically, we argue that the heterogeneity of bi-temporal RS images leads to the failure of the change detection model in some cases. Therefore, we attempt to achieve feature alignment in two aspects by feature exchange. On one hand, we exchange the statistical information of bi-temporal features, facilitating the transfer of their style. On the other hand, bi-temporal features are composed of content correlation embeddings (CCEs) and domain correlation embeddings (DCEs). We design a dynamic low-pass filter (DLF) to partially extract and exchange DCEs in bi-temporal features to achieve the feature distribution alignment. Moreover, a frequency separation enhancement module (FSEM) is proposed, which transforms the fused features into the frequency domain and enhances the corresponding frequency representation. Comprehensive experimental results on three popular CD datasets demonstrate the effectiveness and efficiency of ExNet compared with state-of-the-art methods. The source code will be accessible at https://github.com/liuyk97/ExNet.
- 13 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 17 (19391404) , 11761-11776
Open AccessExploring the Cross-Temporal Interaction: Feature Exchange and Enhancement for Remote Sensing Change Detection
Change detection (CD) aims to identify surface changes from bi-temporal remote sensing (RS) images, which is a crucial and challenging topic in RS. In recent years, RS images CD has achieved significant advancements through the use of convolutional neural networks (CNNs). However, existing deep learning-based CD methods still face some challenges such as blurry boundaries, pseudo changes caused by fluctuations in imaging conditions, and complexity of change objects in the scene. In this study, we present a novel perspective for exploring the cross-temporal interaction in the feature fusion stage and propose a bi-temporal feature exchange and enhancement network (ExNet). Specifically, we argue that the heterogeneity of bi-temporal RS images leads to the failure of the change detection model in some cases. Therefore, we attempt to achieve feature alignment in two aspects by feature exchange. On one hand, we exchange the statistical information of bi-temporal features, facilitating the transfer of their style. On the other hand, bi-temporal features are composed of content correlation embeddings (CCEs) and domain correlation embeddings (DCEs). We design a dynamic low-pass filter (DLF) to partially extract and exchange DCEs in bi-temporal features to achieve the feature distribution alignment. Moreover, a frequency separation enhancement module (FSEM) is proposed, which transforms the fused features into the frequency domain and enhances the corresponding frequency representation. Comprehensive experimental results on three popular CD datasets demonstrate the effectiveness and efficiency of ExNet compared with state-of-the-art methods. The source code will be accessible at https://github.com/liuyk97/ExNet.
- 1 January 2024
- Published by SPIE-Intl Soc Optical Eng in Journal of Applied Remote Sensing
- Vol. 18 (01) , 016513-18
Multilevel feature aggregation and enhancement network for remote sensing change detection
Remote sensing change detection refers to the process of identifying and extracting changes in objects within the same geographical region over multiple periods. With the increasing spatial resolution of remote sensing images, the detection of minor changes has become a challenging task. We introduce a multilevel feature aggregation and enhancement network to tackle this issue. Specifically, we propose a multilevel feature aggregation module to aggregate the distinct features extracted from each image, which strengthens the feature representation capability. Subsequently, a difference parallel mapping module is designed to perceive information at different scales by refining the fused features. In addition, our guided change enhancement module captures local and long-range dependencies in multilevel features, improving the network’s accuracy in identifying changing regions. Based on a basic shared weight Siamese backbone without complex structures, our model outperforms other state-of-the-art methods on three datasets in terms of both efficiency and effectiveness.
- 1 January 2024
- Published by SPIE-Intl Soc Optical Eng in Journal of Applied Remote Sensing
- Vol. 18 (01) , 016513-18
Multilevel feature aggregation and enhancement network for remote sensing change detection
Remote sensing change detection refers to the process of identifying and extracting changes in objects within the same geographical region over multiple periods. With the increasing spatial resolution of remote sensing images, the detection of minor changes has become a challenging task. We introduce a multilevel feature aggregation and enhancement network to tackle this issue. Specifically, we propose a multilevel feature aggregation module to aggregate the distinct features extracted from each image, which strengthens the feature representation capability. Subsequently, a difference parallel mapping module is designed to perceive information at different scales by refining the fused features. In addition, our guided change enhancement module captures local and long-range dependencies in multilevel features, improving the network’s accuracy in identifying changing regions. Based on a basic shared weight Siamese backbone without complex structures, our model outperforms other state-of-the-art methods on three datasets in terms of both efficiency and effectiveness.
- 12 December 2023
- Published by Taylor & Francis in International Journal of Remote Sensing
- Vol. 44 (24) , 7827-7856
Feature alignment and refinement for Remote Sensing images change Detection
- 12 December 2023
- Published by Taylor & Francis in International Journal of Remote Sensing
- Vol. 44 (24) , 7827-7856
Feature alignment and refinement for Remote Sensing images change Detection
- 7 April 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 61 (01962892) , 1-18
Adaptive Mask Sampling and Manifold to Euclidean Subspace Learning With Distance Covariance Representation for Hyperspectral Image Classification
For the abundant spectral and spatial information recorded in hyperspectral images (HSIs), fully exploring spectral-spatial relationships has attracted widespread attention in hyperspectral image classification (HSIC) community. However, there are still some intractable obstructs. For one thing, in the patch based processing pattern, some spatial neighbor pixels are often inconsistent with the central pixel in land-cover class. For another thing, linear and nonlinear correlations between different spectral bands are vital yet tough for representing and excavating. To overcome these mentioned issues, an adaptive mask sampling and manifold to Euclidean subspace learning (AMS-M2ESL) framework is proposed for HSIC. Specifically, an adaptive mask based intra-patch sampling (AMIPS) module is firstly formulated for intra-patch sampling in an adaptive mask manner based on central spectral vector oriented spatial relationships. Then, based on distance covariance descriptor, a dual channel distance covariance representation (DC-DCR) module is proposed for modeling unified spectral-spatial feature representations and exploring spectral-spatial relationships, especially linear and nonlinear interdependence in spectral domain. Furthermore, considering that distance covariance matrix lies on the symmetric positive definite (SPD) manifold, we implement a manifold to Euclidean subspace learning (M2ESL) module respecting Riemannian geometry of SPD manifold for high-level spectral-spatial feature learning. Additionally, we introduce an approximate matrix square-root (ASQRT) layer for efficient Euclidean subspace projection. Extensive experimental results on three popular HSI data sets with limited training samples demonstrate the superior performance of the proposed method compared with other state-of-the-art methods. The source code is available at https://github.com/lms-07/AMS-M2ESL.
- 7 April 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 61 (01962892) , 1-18
Adaptive Mask Sampling and Manifold to Euclidean Subspace Learning With Distance Covariance Representation for Hyperspectral Image Classification
For the abundant spectral and spatial information recorded in hyperspectral images (HSIs), fully exploring spectral-spatial relationships has attracted widespread attention in hyperspectral image classification (HSIC) community. However, there are still some intractable obstructs. For one thing, in the patch based processing pattern, some spatial neighbor pixels are often inconsistent with the central pixel in land-cover class. For another thing, linear and nonlinear correlations between different spectral bands are vital yet tough for representing and excavating. To overcome these mentioned issues, an adaptive mask sampling and manifold to Euclidean subspace learning (AMS-M2ESL) framework is proposed for HSIC. Specifically, an adaptive mask based intra-patch sampling (AMIPS) module is firstly formulated for intra-patch sampling in an adaptive mask manner based on central spectral vector oriented spatial relationships. Then, based on distance covariance descriptor, a dual channel distance covariance representation (DC-DCR) module is proposed for modeling unified spectral-spatial feature representations and exploring spectral-spatial relationships, especially linear and nonlinear interdependence in spectral domain. Furthermore, considering that distance covariance matrix lies on the symmetric positive definite (SPD) manifold, we implement a manifold to Euclidean subspace learning (M2ESL) module respecting Riemannian geometry of SPD manifold for high-level spectral-spatial feature learning. Additionally, we introduce an approximate matrix square-root (ASQRT) layer for efficient Euclidean subspace projection. Extensive experimental results on three popular HSI data sets with limited training samples demonstrate the superior performance of the proposed method compared with other state-of-the-art methods. The source code is available at https://github.com/lms-07/AMS-M2ESL.
- 10 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 61 (01962892) , 1-16
Enhancing Multiscale Representations With Transformer for Remote Sensing Image Semantic Segmentation
Semantic segmentation is an extremely challenging task in high-resolution remote sensing (HRRS) images as objects have complex spatial layouts and enormous variations in appearance. Convolutional neural networks (CNNs) have excellent ability to extract local features and have been widely applied as the feature extractor for various vision tasks. However, due to the inherent inductive bias of convolution operation, CNNs inevitably have limitations in modeling long-range dependencies. Transformer can capture global representations well, but unfortunately ignores the details of local features and has high computational and spatial complexity in processing high-resolution feature maps. In this paper, we propose a novel hybrid architecture for HRRS image segmentation, termed EMRT, to exploit the advantages of convolution operations and Transformer to enhance multi-scale representation learning. We incorporate the deformable self-attention mechanism in the Transformer to automatically adjust the receptive field, and design an encoder-decoder architecture accordingly to achieve efficient context modeling. Specifically, the CNN is constructed to extract feature representations. In the encoder, local features and global representations at different resolutions are extracted by the CNN and Transformer, respectively, and fused in an interactive manner. Moreover, a separate spatial branch is designed to extract multi-scale contextual information as queries, and global dependencies between features at different scales are efficiently established by the decoder. Extensive experiments on three public remote sensing datasets demonstrate the superiority of EMRT and indicate that the overall performance of our method outperforms state-of-the-art methods. Code is available at https://github.com/peach-xiao/EMRT.
- 10 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 61 (01962892) , 1-16
Enhancing Multiscale Representations With Transformer for Remote Sensing Image Semantic Segmentation
Semantic segmentation is an extremely challenging task in high-resolution remote sensing (HRRS) images as objects have complex spatial layouts and enormous variations in appearance. Convolutional neural networks (CNNs) have excellent ability to extract local features and have been widely applied as the feature extractor for various vision tasks. However, due to the inherent inductive bias of convolution operation, CNNs inevitably have limitations in modeling long-range dependencies. Transformer can capture global representations well, but unfortunately ignores the details of local features and has high computational and spatial complexity in processing high-resolution feature maps. In this paper, we propose a novel hybrid architecture for HRRS image segmentation, termed EMRT, to exploit the advantages of convolution operations and Transformer to enhance multi-scale representation learning. We incorporate the deformable self-attention mechanism in the Transformer to automatically adjust the receptive field, and design an encoder-decoder architecture accordingly to achieve efficient context modeling. Specifically, the CNN is constructed to extract feature representations. In the encoder, local features and global representations at different resolutions are extracted by the CNN and Transformer, respectively, and fused in an interactive manner. Moreover, a separate spatial branch is designed to extract multi-scale contextual information as queries, and global dependencies between features at different scales are efficiently established by the decoder. Extensive experiments on three public remote sensing datasets demonstrate the superiority of EMRT and indicate that the overall performance of our method outperforms state-of-the-art methods. Code is available at https://github.com/peach-xiao/EMRT.
- Top Cited Papers
- 31 October 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 33 (4) , 1979-1993
Exploring the Relationship Between Center and Neighborhoods: Central Vector Oriented Self-Similarity Network for Hyperspectral Image Classification
To mine the spectral-spatial information of target pixel in hyperspectral image classification (HSIC), convolutional neural network (CNN)-based models widely adopt patch-based input pattern, where a patch represents its central pixel and the neighbor pixels play auxiliary roles in the classification process. However, compared to the central pixel, its neighbor pixels often have different contributions for classification. Although many existing patch-based CNNs could adaptively emphasize the spatial neighbor information, most of them ignore the latent relationship between the center pixel and its neighbor pixels. Moreover, efficient spectral-spatial feature extraction has been a difficult yet vital topic for HSIC. To address the mentioned problems, a central vector oriented self-similarity network (CVSSN) is proposed for HSIC. Specifically, based on two similarity measures, we firstly design an adaptive weight addition based spectral vector self-similarity module (AWA-SVSS) in input space and a Euclidean distance based feature vector self-similarity module (ED-FVSS) in feature space to fully mine the central vector oriented spatial relationships. Besides, a spectral-spatial information fusion module (SSIF) is formulated as a new pattern to fuse the central 1D spectral vector and the corresponding 3D patch for efficient spectral-spatial feature learning of the subsequent modules. Moreover, we implement a channel spatial separation convolution module (CSS-Conv) and a scale information complementary convolution module (SIC-Conv) for efficient spectral-spatial feature learning. Extensive experimental results on four popular HSI data sets demonstrate the effectiveness and efficiency of the proposed method compared with other state-of-the-art methods.
- 31 October 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 33 (4) , 1979-1993
Top Cited PapersExploring the Relationship Between Center and Neighborhoods: Central Vector Oriented Self-Similarity Network for Hyperspectral Image Classification
To mine the spectral-spatial information of target pixel in hyperspectral image classification (HSIC), convolutional neural network (CNN)-based models widely adopt patch-based input pattern, where a patch represents its central pixel and the neighbor pixels play auxiliary roles in the classification process. However, compared to the central pixel, its neighbor pixels often have different contributions for classification. Although many existing patch-based CNNs could adaptively emphasize the spatial neighbor information, most of them ignore the latent relationship between the center pixel and its neighbor pixels. Moreover, efficient spectral-spatial feature extraction has been a difficult yet vital topic for HSIC. To address the mentioned problems, a central vector oriented self-similarity network (CVSSN) is proposed for HSIC. Specifically, based on two similarity measures, we firstly design an adaptive weight addition based spectral vector self-similarity module (AWA-SVSS) in input space and a Euclidean distance based feature vector self-similarity module (ED-FVSS) in feature space to fully mine the central vector oriented spatial relationships. Besides, a spectral-spatial information fusion module (SSIF) is formulated as a new pattern to fuse the central 1D spectral vector and the corresponding 3D patch for efficient spectral-spatial feature learning of the subsequent modules. Moreover, we implement a channel spatial separation convolution module (CSS-Conv) and a scale information complementary convolution module (SIC-Conv) for efficient spectral-spatial feature learning. Extensive experimental results on four popular HSI data sets demonstrate the effectiveness and efficiency of the proposed method compared with other state-of-the-art methods.
- 3 July 2022
- Published by Taylor & Francis in International Journal of Remote Sensing
- Vol. 43 (13) , 4836-4852
AANet: an attention-based alignment semantic segmentation network for high spatial resolution remote sensing images
In this paper, we present an efficient network to tackle three critical problems in high spatial resolution (HSR) remote sensing image segmentation: feature misalignment, insufficient contextual information extraction and various class imbalance issues. In detail, we propose a novel Feature Alignment Block (FAB) to suppress misalignment issues with the guide of an anchor map. Further, to extract sufficient information, we design a Contextual Augmentation Block (CAB) to augment features of different semantic levels. Finally, we present an Annealing Online Hard Example Mining (AOHEM) strategy to handle the various class imbalance issues with a view to dynamically adjust the focus of the network. We apply the above proposed designs to FPN to form our Attention-based Alignment Network (AANet). Experimental results demonstrate that the proposed method achieves promising results on the challenging iSAID and Vaihingen datasets with a better trade-off between accuracy and complexity.- 3 July 2022
- Published by Taylor & Francis in International Journal of Remote Sensing
- Vol. 43 (13) , 4836-4852
AANet: an attention-based alignment semantic segmentation network for high spatial resolution remote sensing images
In this paper, we present an efficient network to tackle three critical problems in high spatial resolution (HSR) remote sensing image segmentation: feature misalignment, insufficient contextual information extraction and various class imbalance issues. In detail, we propose a novel Feature Alignment Block (FAB) to suppress misalignment issues with the guide of an anchor map. Further, to extract sufficient information, we design a Contextual Augmentation Block (CAB) to augment features of different semantic levels. Finally, we present an Annealing Online Hard Example Mining (AOHEM) strategy to handle the various class imbalance issues with a view to dynamically adjust the focus of the network. We apply the above proposed designs to FPN to form our Attention-based Alignment Network (AANet). Experimental results demonstrate that the proposed method achieves promising results on the challenging iSAID and Vaihingen datasets with a better trade-off between accuracy and complexity.- Open Access
- 29 June 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 15 (19391404) , 5607-5622
$\text{A}^{3}$Seg: An Annealing Augmented and Aligned Semantic Segmentation Network for High Spatial Resolution Remote Sensing Images
Semantic segmentation of high spatial resolution (HSR) remote sensing images (RSIs) plays an important role in many applications. However, HSR RSIs have significantly larger spatial sizes than typical natural images, which results in fewer valuable samples when training models. In addition, fusing multiscale features is the key step in obtaining features with strong semantic and high spatial information. However, current feature fusion methods are too straightforward to address misalignment issues. To handle these two problems, we propose an annealing augmented and aligned segmentation network, named $\text{A}^{3}$ Seg. Specifically, we propose an annealing online hard example mining (AOHEM) strategy to automatically select more valuable samples during the training stage. Based on AOHEM, a contextual augmentation block is proposed to extract sufficient contextual information using three different attention mechanisms in consideration of three different feature properties. Finally, we propose a novel feature alignment block to fuse features at different levels by alignment with the guidance of a salient feature. Experimental results on three different HSR RSIs datasets demonstrate that the proposed method outperforms the state-of-the-art general semantic segmentation methods with a better tradeoff between accuracy and complexity.
- 29 June 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 15 (19391404) , 5607-5622
Open Access$\text{A}^{3}$Seg: An Annealing Augmented and Aligned Semantic Segmentation Network for High Spatial Resolution Remote Sensing Images
Semantic segmentation of high spatial resolution (HSR) remote sensing images (RSIs) plays an important role in many applications. However, HSR RSIs have significantly larger spatial sizes than typical natural images, which results in fewer valuable samples when training models. In addition, fusing multiscale features is the key step in obtaining features with strong semantic and high spatial information. However, current feature fusion methods are too straightforward to address misalignment issues. To handle these two problems, we propose an annealing augmented and aligned segmentation network, named $\text{A}^{3}$ Seg. Specifically, we propose an annealing online hard example mining (AOHEM) strategy to automatically select more valuable samples during the training stage. Based on AOHEM, a contextual augmentation block is proposed to extract sufficient contextual information using three different attention mechanisms in consideration of three different feature properties. Finally, we propose a novel feature alignment block to fuse features at different levels by alignment with the guidance of a salient feature. Experimental results on three different HSR RSIs datasets demonstrate that the proposed method outperforms the state-of-the-art general semantic segmentation methods with a better tradeoff between accuracy and complexity.