Shouhang Du
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Shouhang Du has a total of 81 co-authors. Here are the top co-authors by number of shared publications:
- Shihong Du
Institute of Remote Sensing and GIS, Peking University, Beijing, China
7 shared publications - Jun Li
College of Geoscience and Surveying Engineering, China University of Mining and Technology (Beijing), Beijing 100083, China
6 shared publications - Jianghe Xing
College of Geoscience and Surveying Engineering, China University of Mining and Technology (Beijing), Beijing, China
5 shared publications - Xiuyuan Zhang
College of Urban and Environmental Sciences, Peking University, Beijing, China
5 shared publications - Chengye Zhang
College of Geoscience and Surveying Engineering, China University of Mining and Technology (Beijing), Beijing 100083, China
4 shared publications Deqin Fan
School of Geoscience and Surveying Engineering, China University of Mining and Technology (Beijing), Beijing, China
4 shared publications
Latest Publications
- Open Access
- 27 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 19 (19391404) , 8818-8837
Layered Scene-Aware Network: Fusing Multimodal Data for Fine-Grained Urban Functional Zone Classification
As the fundamental spatial units of urban areas, urban functional zones require accurate classification to support urban planning, spatial structure monitoring, and sustainable development research. However, existing studies have insufficiently explored scene-related features within multimodal data, and most research focuses solely on coarse-grained classification systems, making it difficult to achieve a refined understanding of internal urban spatial structures. To address this issue, this study proposes a Layered Scene-Aware Network (LSANet) for fine-grained urban functional zone classification. This network integrates scene visual features from remote sensing images, scene semantic features from point of interest (POI) textual information, and scene spatial features from POI spatial distribution information, constructing a multi-level representation of multimodal data correlation features. A dynamic attention mechanism is further introduced to enhance cross-modal interactions and improve classification accuracy and robustness. This study selected the area within Beijing's Sixth Ring Road as the case study. Results demonstrate that the proposed network achieves an overall accuracy of 85.76% and a Kappa coefficient of 0.8208 on the dataset. In addition, experiments on an independent Shanghai study area further validate the effectiveness and generalization capability of the proposed framework. Moreover, the scenario-based information complexity index constructed in this study can effectively measure the spatial heterogeneity and semantic diversity of different functional zones, providing methodological support for urban spatial cognition and refined governance.
- 27 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 19 (19391404) , 8818-8837
Open AccessLayered Scene-Aware Network: Fusing Multimodal Data for Fine-Grained Urban Functional Zone Classification
As the fundamental spatial units of urban areas, urban functional zones require accurate classification to support urban planning, spatial structure monitoring, and sustainable development research. However, existing studies have insufficiently explored scene-related features within multimodal data, and most research focuses solely on coarse-grained classification systems, making it difficult to achieve a refined understanding of internal urban spatial structures. To address this issue, this study proposes a Layered Scene-Aware Network (LSANet) for fine-grained urban functional zone classification. This network integrates scene visual features from remote sensing images, scene semantic features from point of interest (POI) textual information, and scene spatial features from POI spatial distribution information, constructing a multi-level representation of multimodal data correlation features. A dynamic attention mechanism is further introduced to enhance cross-modal interactions and improve classification accuracy and robustness. This study selected the area within Beijing's Sixth Ring Road as the case study. Results demonstrate that the proposed network achieves an overall accuracy of 85.76% and a Kappa coefficient of 0.8208 on the dataset. In addition, experiments on an independent Shanghai study area further validate the effectiveness and generalization capability of the proposed framework. Moreover, the scenario-based information complexity index constructed in this study can effectively measure the spatial heterogeneity and semantic diversity of different functional zones, providing methodological support for urban spatial cognition and refined governance.
- Open Access
- 26 December 2025
- Published by MDPI AG in Applied Sciences
- Vol. 16 (1) , 251
Integrating CNN-Mamba and Frequency-Domain Information for Urban Scene Classification from High-Resolution Remote Sensing Images
Urban scene classification in high-resolution remote sensing images is critical for applications such as power facility site selection and grid security monitoring. However, the complexity and variability of ground objects present significant challenges to accurate classification. While convolutional neural networks (CNNs) excel at extracting local features, they often struggle to model long-range dependencies. Transformers can capture global context but incur high computational costs. To address these limitations, this paper proposes a Global–Local Information Fusion Network (GLIFNet), which integrates VMamba for efficient global modeling with CNN for local detail extraction, enabling more effective fusion of fine-grained and semantic information. Furthermore, a Haar Wavelet Transform Attention Mechanism (HWTAM) is designed to explicitly exploit frequency-domain characteristics, facilitating refined fusion of multi-scale features. The experiment compared nine commonly used or most advanced methods. The results show that GLIFNet achieves mean F1 scores (mF1) of 90.08% and 87.44% on the ISPRS Potsdam and ISPRS Vaihingen datasets, respectively. This represents improvements of 1.26% and 1.91%, respectively, compared to the compared model. The overall accuracy (OA) reaches 90.43% and 92.87%, with respective gains of 2.28% and 1.58%. Experimental results on the LandCover.ai dataset demonstrate that GLIFNet achieved an mF1 score of 88.39% and an accuracy of 92.23%, exhibiting relative improvements of 0.3% and 0.28% compared with the control model. In summary, GLIFNet demonstrates advanced performance in urban scene classification from high-resolution remote sensing images and can provide accurate basic data for power construction.
- 26 December 2025
- Published by MDPI AG in Applied Sciences
- Vol. 16 (1) , 251
Open AccessIntegrating CNN-Mamba and Frequency-Domain Information for Urban Scene Classification from High-Resolution Remote Sensing Images
Urban scene classification in high-resolution remote sensing images is critical for applications such as power facility site selection and grid security monitoring. However, the complexity and variability of ground objects present significant challenges to accurate classification. While convolutional neural networks (CNNs) excel at extracting local features, they often struggle to model long-range dependencies. Transformers can capture global context but incur high computational costs. To address these limitations, this paper proposes a Global–Local Information Fusion Network (GLIFNet), which integrates VMamba for efficient global modeling with CNN for local detail extraction, enabling more effective fusion of fine-grained and semantic information. Furthermore, a Haar Wavelet Transform Attention Mechanism (HWTAM) is designed to explicitly exploit frequency-domain characteristics, facilitating refined fusion of multi-scale features. The experiment compared nine commonly used or most advanced methods. The results show that GLIFNet achieves mean F1 scores (mF1) of 90.08% and 87.44% on the ISPRS Potsdam and ISPRS Vaihingen datasets, respectively. This represents improvements of 1.26% and 1.91%, respectively, compared to the compared model. The overall accuracy (OA) reaches 90.43% and 92.87%, with respective gains of 2.28% and 1.58%. Experimental results on the LandCover.ai dataset demonstrate that GLIFNet achieved an mF1 score of 88.39% and an accuracy of 92.23%, exhibiting relative improvements of 0.3% and 0.28% compared with the control model. In summary, GLIFNet demonstrates advanced performance in urban scene classification from high-resolution remote sensing images and can provide accurate basic data for power construction.
- Open Access
A National-Scale Evaluation of Eco-City Development in China: Spatial Heterogeneity, Obstacle Factors, and Relationship with Carbon Intensity
Under the national “dual-carbon goal” and the pressing demand for sustainable development, eco-city construction and carbon reduction have become critical issues on China’s urban development agenda, closely aligned with the United Nations Sustainable Development Goals (SDGs). However, most studies focus on regional assessments, lacking national-scale evaluations and spatial heterogeneity analysis of obstacles. This study analyzes 280 Chinese cities using a multi-level evaluation system. Analytic hierarchy process (AHP) and entropy weight methods determine index weights, while the comprehensive evaluation method assesses ecological levels. The obstacle diagnosis model identifies key obstacle factors, and geographically weighted regression (GWR) analyzes spatial heterogeneity, computing carbon intensity to explore relationships with eco-cities development. The findings reveal that (1) the ecological level of Chinese cities exhibits a regional pattern of “high in the east, low in the west”; (2) the primary index-level obstacle factors include total per capita water resources, per capita green space area, college full-time faculty per 10,000 people, the proportion of tertiary industries in gross domestic product (GDP), and college students per 10,000 people; at the element level, the main obstacles are environmental bases, social services, economic potential, and innovative capacity; (3) the GWR model reveals that eastern regions should increase water resources, central regions expand green space, and western and northeastern regions enhance innovative capacity and social services to foster balanced development; and (4) carbon intensity follows a “low in the east, high in the west” pattern, with eco-cities scores significantly negatively correlated with carbon intensity (r = −0.235, p < 0.01). This study provides the first comprehensive national-scale evaluation of eco-cities development, providing reference for the construction of eco-cities.
Open AccessA National-Scale Evaluation of Eco-City Development in China: Spatial Heterogeneity, Obstacle Factors, and Relationship with Carbon Intensity
Under the national “dual-carbon goal” and the pressing demand for sustainable development, eco-city construction and carbon reduction have become critical issues on China’s urban development agenda, closely aligned with the United Nations Sustainable Development Goals (SDGs). However, most studies focus on regional assessments, lacking national-scale evaluations and spatial heterogeneity analysis of obstacles. This study analyzes 280 Chinese cities using a multi-level evaluation system. Analytic hierarchy process (AHP) and entropy weight methods determine index weights, while the comprehensive evaluation method assesses ecological levels. The obstacle diagnosis model identifies key obstacle factors, and geographically weighted regression (GWR) analyzes spatial heterogeneity, computing carbon intensity to explore relationships with eco-cities development. The findings reveal that (1) the ecological level of Chinese cities exhibits a regional pattern of “high in the east, low in the west”; (2) the primary index-level obstacle factors include total per capita water resources, per capita green space area, college full-time faculty per 10,000 people, the proportion of tertiary industries in gross domestic product (GDP), and college students per 10,000 people; at the element level, the main obstacles are environmental bases, social services, economic potential, and innovative capacity; (3) the GWR model reveals that eastern regions should increase water resources, central regions expand green space, and western and northeastern regions enhance innovative capacity and social services to foster balanced development; and (4) carbon intensity follows a “low in the east, high in the west” pattern, with eco-cities scores significantly negatively correlated with carbon intensity (r = −0.235, p < 0.01). This study provides the first comprehensive national-scale evaluation of eco-cities development, providing reference for the construction of eco-cities.
- 18 October 2025
- Published by Springer Nature in Water, Air, & Soil Pollution
- Vol. 237 (1) , 1-23
Analysis of the Spatial and Temporal Distribution of Fires and their Atmospheric Changes in the Amazon from 2001 to 2022
- 18 October 2025
- Published by Springer Nature in Water, Air, & Soil Pollution
- Vol. 237 (1) , 1-23
Analysis of the Spatial and Temporal Distribution of Fires and their Atmospheric Changes in the Amazon from 2001 to 2022
- 15 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 63 (01962892) , 1-20
Semi-Supervised Semantic Remote Sensing Image Change Detection Using Multimodal Spatiotemporal Association Knowledge
Affected by the cost of producing semantic change detection samples, semi-supervised semantic change detection methods have gradually become a research hotspot. However, existing semi-supervised semantic change detection methods only consider unimodal remote sensing image data, which easily overlooks the rich spatio-temporal semantic information available in the data. This limitation restricts the performance of semi-supervised semantic change detection. To address these issues, this study proposes a multimodal spatio-temporal association knowledge-based semi-supervised semantic change detection (MST-AK) method. First, the MST-AK method introduces the CLIP model for the first time to construct land cover descriptive textual knowledge, overcoming the limitations of unimodal data in semi-supervised semantic change detection. Second, the image-text dual-attention difference change feature fusion module and the three-branch interactive spatio-temporal association module are designed to enable cross-modal learning between different spatio-temporal images from a multimodal perspective. Finally, the change magnitude-based adaptive threshold selection method is used to adaptively select the threshold value based on the distribution of change magnitude in the change feature image, improving the distinguishability of the semantic change detection results. To demonstrate the feasibility of the MST-AK method, two commonly used publicly available semantic change detection datasets, SECOND and Landsat-SCD, are utilized for comparison and validation against state-of-the-art methods. Additionally, areas in Beijing and Yangzhou cities, China, are selected to demonstrate the application of semantic change detection. The results show that the MST-AK method can fully explore the temporal and spatial semantic change features of multimodal remote sensing images, break the limitations of traditional models relying on a single modality, enable cross-modal learning and understanding, and improve the accuracy of semantic change detection results even with fewer samples.
- 15 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 63 (01962892) , 1-20
Semi-Supervised Semantic Remote Sensing Image Change Detection Using Multimodal Spatiotemporal Association Knowledge
Affected by the cost of producing semantic change detection samples, semi-supervised semantic change detection methods have gradually become a research hotspot. However, existing semi-supervised semantic change detection methods only consider unimodal remote sensing image data, which easily overlooks the rich spatio-temporal semantic information available in the data. This limitation restricts the performance of semi-supervised semantic change detection. To address these issues, this study proposes a multimodal spatio-temporal association knowledge-based semi-supervised semantic change detection (MST-AK) method. First, the MST-AK method introduces the CLIP model for the first time to construct land cover descriptive textual knowledge, overcoming the limitations of unimodal data in semi-supervised semantic change detection. Second, the image-text dual-attention difference change feature fusion module and the three-branch interactive spatio-temporal association module are designed to enable cross-modal learning between different spatio-temporal images from a multimodal perspective. Finally, the change magnitude-based adaptive threshold selection method is used to adaptively select the threshold value based on the distribution of change magnitude in the change feature image, improving the distinguishability of the semantic change detection results. To demonstrate the feasibility of the MST-AK method, two commonly used publicly available semantic change detection datasets, SECOND and Landsat-SCD, are utilized for comparison and validation against state-of-the-art methods. Additionally, areas in Beijing and Yangzhou cities, China, are selected to demonstrate the application of semantic change detection. The results show that the MST-AK method can fully explore the temporal and spatial semantic change features of multimodal remote sensing images, break the limitations of traditional models relying on a single modality, enable cross-modal learning and understanding, and improve the accuracy of semantic change detection results even with fewer samples.
- 21 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. PP (01962892) , 1
Estimating Individual Building Heights by Integrating Spaceborne LiDAR and Multisource Remote Sensing Data: A CNN-Transformer Model and A Semi-Supervised Sample Augmentation Approach
Building height is a key metric in transitioning urban analysis from two-dimensional to three-dimensional perspectives. It serves as a fundamental indicator for assessing urban development, population density, and energy consumption. Regardless of whether machine learning or deep learning methods are used, accurate height estimation inevitably relies on large quantities of labeled building height samples for model training. However, existing reference datasets often suffer from ambiguity due to floor-to-height conversions and temporal obsolescence. The advent of spaceborne light detection and ranging (LiDAR) provides a promising avenue to overcome these limitations, enabling the acquisition of large-scale, high-precision, and time-sensitive reference height data. Moreover, while buildings are inherently individual and discrete units, most existing height estimation studies operate at gridded scales, thereby obscuring inter-building height variability. In this study, we propose a novel approach to estimate individual building heights by integrating spaceborne LiDAR with multisource remote sensing data. First, leveraging building footprint data, we derive reference height samples by jointly retrieving building heights from the Ice, Cloud, and land Elevation Satellite-2 (ICESat-2) and the Global Ecosystem Dynamics Investigation (GEDI). Then, we construct a multimodal feature set by extracting seasonal features from a combination of multisource datasets, such as Sentinel-1, Sentinel-2, and SDGSAT-1. To model the spatially complex height distribution, we propose the Neighborhood-aware CNN-Transformer Network (NeCT-Net), which estimates building heights at the pixel level. To address the scarcity of tall building samples, we incorporate a semi-supervised learning paradigm. Specifically, pseudo-labels for high-rise buildings are generated using predictions from a teacher model and iteratively refined through self-training, thereby enhancing the student model’s learning capacity in tall-structure scenarios. Finally, we propose a post-processing approach that constrains height predictions using building morphological information, enabling the conversion from pixel-level predictions to vector-based building-level estimations. We apply the proposed method to two iconic urban areas—Beijing and Seattle. The results demonstrate strong predictive performance, with RMSE, MAE, and R values of 9.01 m, 6.08 m, and 0.93 for Beijing, and 8.72 m, 5.31 m, and 0.79 for Seattle. Compared to baseline methods, our approach reduces RMSE by 29.66% in Beijing and 7.43% in Seattle, while improving R by 47.62% and 46.06%. When benchmarked against existing individual building height products, our model achieves RMSE reductions of 31.43% and 21.16%, and R improvements of 57.63% and 23.44% in the two cities. These results highlight the methodological advancement and robustness of our approach. This work presents a new paradigm for large-scale, individual building height estimation through the integration of spaceborne LiDAR and multisource remote sensing data. The source code will be made available at https://github.com/Lunar-Elf/NeCT-Net.
- 21 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. PP (01962892) , 1
Estimating Individual Building Heights by Integrating Spaceborne LiDAR and Multisource Remote Sensing Data: A CNN-Transformer Model and A Semi-Supervised Sample Augmentation Approach
Building height is a key metric in transitioning urban analysis from two-dimensional to three-dimensional perspectives. It serves as a fundamental indicator for assessing urban development, population density, and energy consumption. Regardless of whether machine learning or deep learning methods are used, accurate height estimation inevitably relies on large quantities of labeled building height samples for model training. However, existing reference datasets often suffer from ambiguity due to floor-to-height conversions and temporal obsolescence. The advent of spaceborne light detection and ranging (LiDAR) provides a promising avenue to overcome these limitations, enabling the acquisition of large-scale, high-precision, and time-sensitive reference height data. Moreover, while buildings are inherently individual and discrete units, most existing height estimation studies operate at gridded scales, thereby obscuring inter-building height variability. In this study, we propose a novel approach to estimate individual building heights by integrating spaceborne LiDAR with multisource remote sensing data. First, leveraging building footprint data, we derive reference height samples by jointly retrieving building heights from the Ice, Cloud, and land Elevation Satellite-2 (ICESat-2) and the Global Ecosystem Dynamics Investigation (GEDI). Then, we construct a multimodal feature set by extracting seasonal features from a combination of multisource datasets, such as Sentinel-1, Sentinel-2, and SDGSAT-1. To model the spatially complex height distribution, we propose the Neighborhood-aware CNN-Transformer Network (NeCT-Net), which estimates building heights at the pixel level. To address the scarcity of tall building samples, we incorporate a semi-supervised learning paradigm. Specifically, pseudo-labels for high-rise buildings are generated using predictions from a teacher model and iteratively refined through self-training, thereby enhancing the student model’s learning capacity in tall-structure scenarios. Finally, we propose a post-processing approach that constrains height predictions using building morphological information, enabling the conversion from pixel-level predictions to vector-based building-level estimations. We apply the proposed method to two iconic urban areas—Beijing and Seattle. The results demonstrate strong predictive performance, with RMSE, MAE, and R values of 9.01 m, 6.08 m, and 0.93 for Beijing, and 8.72 m, 5.31 m, and 0.79 for Seattle. Compared to baseline methods, our approach reduces RMSE by 29.66% in Beijing and 7.43% in Seattle, while improving R by 47.62% and 46.06%. When benchmarked against existing individual building height products, our model achieves RMSE reductions of 31.43% and 21.16%, and R improvements of 57.63% and 23.44% in the two cities. These results highlight the methodological advancement and robustness of our approach. This work presents a new paradigm for large-scale, individual building height estimation through the integration of spaceborne LiDAR and multisource remote sensing data. The source code will be made available at https://github.com/Lunar-Elf/NeCT-Net.
- 11 April 2025
- Published by Elsevier in Energy and Buildings
- Vol. 338
Multi-scale impact of urban building function and 2D/3D morphology on urban heat island effect: a case study in Shanghai, China
- 11 April 2025
- Published by Elsevier in Energy and Buildings
- Vol. 338
Multi-scale impact of urban building function and 2D/3D morphology on urban heat island effect: a case study in Shanghai, China
- Open Access
Sub-District Level Spatiotemporal Changes of Carbon Storage and Driving Factor Analysis: A Case Study in Beijing
Analyzing the current trends and causes of carbon storage changes and accurately predicting future land use and carbon storage changes under different climate scenarios is crucial for regional land use decision-making and carbon management. This study focuses on Beijing as its study area and introduces a framework that combines the Markov model, the Patch-based Land Use Simulation (PLUS) model, and the Integrated Valuation of Ecosystem Services and Tradeoffs (InVEST) model to assess carbon storage at the sub-district level. This framework allows for a systematic analysis of land use and carbon storage spatiotemporal evolution in Beijing from 2000 to 2020, including the influence of driving factors on carbon storage. Moreover, it enables the simulation and prediction of land use and carbon storage changes in Beijing from 2025 to 2040 under various scenarios. The results show the following: (1) From 2000 to 2020, the overall land use change in Beijing showed a trend of “Significant decrease in cropland area; Forest increase gradually; Shrub and grassland area increase first and then decrease; Decrease and then increase in water; Impervious expands in a large scale”. (2) From 2000 to 2020, the carbon storage in Beijing showed a “decrease-increase” fluctuation, with an overall decrease of 1.3 Tg. In future carbon storage prediction, the ecological protection scenario will contribute to achieving the goals of carbon peak and carbon neutrality. (3) Among the various driving factors, slope has the strongest impact on the overall carbon storage in Beijing, followed by Human Activity Intensity (HAI) and Nighttime Light Data (NTL). In the analysis of carbon storage in the built-up areas, it was found that HAI and DEM (Digital Elevation Model) have the strongest effect, followed by NTL and Fractional Vegetation Cover (FVC). The findings from this study offer valuable insights for the sustainable advancement of ecological conservation and urban development in Beijing.
Open AccessSub-District Level Spatiotemporal Changes of Carbon Storage and Driving Factor Analysis: A Case Study in Beijing
Analyzing the current trends and causes of carbon storage changes and accurately predicting future land use and carbon storage changes under different climate scenarios is crucial for regional land use decision-making and carbon management. This study focuses on Beijing as its study area and introduces a framework that combines the Markov model, the Patch-based Land Use Simulation (PLUS) model, and the Integrated Valuation of Ecosystem Services and Tradeoffs (InVEST) model to assess carbon storage at the sub-district level. This framework allows for a systematic analysis of land use and carbon storage spatiotemporal evolution in Beijing from 2000 to 2020, including the influence of driving factors on carbon storage. Moreover, it enables the simulation and prediction of land use and carbon storage changes in Beijing from 2025 to 2040 under various scenarios. The results show the following: (1) From 2000 to 2020, the overall land use change in Beijing showed a trend of “Significant decrease in cropland area; Forest increase gradually; Shrub and grassland area increase first and then decrease; Decrease and then increase in water; Impervious expands in a large scale”. (2) From 2000 to 2020, the carbon storage in Beijing showed a “decrease-increase” fluctuation, with an overall decrease of 1.3 Tg. In future carbon storage prediction, the ecological protection scenario will contribute to achieving the goals of carbon peak and carbon neutrality. (3) Among the various driving factors, slope has the strongest impact on the overall carbon storage in Beijing, followed by Human Activity Intensity (HAI) and Nighttime Light Data (NTL). In the analysis of carbon storage in the built-up areas, it was found that HAI and DEM (Digital Elevation Model) have the strongest effect, followed by NTL and Fractional Vegetation Cover (FVC). The findings from this study offer valuable insights for the sustainable advancement of ecological conservation and urban development in Beijing.
- 6 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 62 (01962892) , 1-19
CTMNet: Enhanced Open-Pit Mine Extraction and Change Detection With a Hybrid CNN-Transformer Multitask Network
Automatic open-pit mine extraction and change detection from high-resolution remote sensing images are of great importance to mineral resource management. However, the high spatial heterogeneity and spectral variations of mining areas scenarios makes these tasks challenging. Motivated by the strong correlation between the two tasks and their potential mutual benefits, this paper presents a hybrid convolutional neural network (CNN)-transformer multitask network (CTMNet). Constructed in an Encoder-Decoder manner, CTMNet has two sperate extraction paths (EPs) to localize the regions of interest for bi-temporal images, along with a change detection path (CDP) to identify discrepancies by differentiate the multi-scale feature representations from the EPs. As the basic building block for the EP, a CNN-Transformer hybrid block is designed to enhance the global and local feature representation capacity. To cope with the variations in the bi-temporal images, we propose the feature alignment module for the CDP. A hard sample mining based contrastive constraint loss is proposed to emphasize the contributions of hard samples to the training process. Experimental results on a collected open-pit mine extraction and change detection dataset (OMECSet) and two public datasets reveal the validity of the CTMNet when compared to the state-of-the-art methods. The OMECSet and the code of CTMNet have been made public available at: https://figshare.com/s/80519cb980ca54456447.
- 6 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 62 (01962892) , 1-19
CTMNet: Enhanced Open-Pit Mine Extraction and Change Detection With a Hybrid CNN-Transformer Multitask Network
Automatic open-pit mine extraction and change detection from high-resolution remote sensing images are of great importance to mineral resource management. However, the high spatial heterogeneity and spectral variations of mining areas scenarios makes these tasks challenging. Motivated by the strong correlation between the two tasks and their potential mutual benefits, this paper presents a hybrid convolutional neural network (CNN)-transformer multitask network (CTMNet). Constructed in an Encoder-Decoder manner, CTMNet has two sperate extraction paths (EPs) to localize the regions of interest for bi-temporal images, along with a change detection path (CDP) to identify discrepancies by differentiate the multi-scale feature representations from the EPs. As the basic building block for the EP, a CNN-Transformer hybrid block is designed to enhance the global and local feature representation capacity. To cope with the variations in the bi-temporal images, we propose the feature alignment module for the CDP. A hard sample mining based contrastive constraint loss is proposed to emphasize the contributions of hard samples to the training process. Experimental results on a collected open-pit mine extraction and change detection dataset (OMECSet) and two public datasets reveal the validity of the CTMNet when compared to the state-of-the-art methods. The OMECSet and the code of CTMNet have been made public available at: https://figshare.com/s/80519cb980ca54456447.
- Open Access
- 23 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 17 (19391404) , 13169-13187
STMNet: Scene Classification-Assisted and Texture Feature-Enhanced Multiscale Network for Large-Scale Urban Informal Settlement Extraction From Remote Sensing Images
Automatic urban informal settlement (UIS) extraction based on high-resolution remote sensing image (HRI) is of great significance for urban planning and management. This study proposes a scene classification-assisted and texture feature-enhanced multi-scale network (STMNet) for UIS extraction. First, STMNet takes HRI and computed handcrafted texture feature (HTF) as input. Second, it employs a pseudo-siamese network to extract multi-dimensional deep features from HRI and HTF, respectively. Additionally, a feature attention fusion module is constructed to fuse the aforementioned features. Finally, skip connection and feature decoder are utilized to obtain UIS extraction results. In detail, considering the sparse and dispersed distribution of UIS, a scene information aggregation and classification module is constructed to determine whether the input image patch contains UIS. For the characteristics of high spatial heterogeneity and various shape and scale of UIS, an improved atrous spatial pyramid pooling is presented to extract multi-scale and multi-receptive field features. An edge loss function is applied during network training to minimize errors in the edge regions of UIS. The effectiveness of STMNet is tested on a self-produced UIS extraction dataset and the publicly available UIS-Shenzhen dataset. Quantitative results demonstrate that STMNet achieved the best performance in terms of fa, F 1, and IoU .The mr is slightly higher than that of MAResU-Net and UisNet. Additionally, STMNet achieved the best visual interpretation results and the fastest inference speed on the self-produced UIS extraction dataset
- 23 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 17 (19391404) , 13169-13187
Open AccessSTMNet: Scene Classification-Assisted and Texture Feature-Enhanced Multiscale Network for Large-Scale Urban Informal Settlement Extraction From Remote Sensing Images
Automatic urban informal settlement (UIS) extraction based on high-resolution remote sensing image (HRI) is of great significance for urban planning and management. This study proposes a scene classification-assisted and texture feature-enhanced multi-scale network (STMNet) for UIS extraction. First, STMNet takes HRI and computed handcrafted texture feature (HTF) as input. Second, it employs a pseudo-siamese network to extract multi-dimensional deep features from HRI and HTF, respectively. Additionally, a feature attention fusion module is constructed to fuse the aforementioned features. Finally, skip connection and feature decoder are utilized to obtain UIS extraction results. In detail, considering the sparse and dispersed distribution of UIS, a scene information aggregation and classification module is constructed to determine whether the input image patch contains UIS. For the characteristics of high spatial heterogeneity and various shape and scale of UIS, an improved atrous spatial pyramid pooling is presented to extract multi-scale and multi-receptive field features. An edge loss function is applied during network training to minimize errors in the edge regions of UIS. The effectiveness of STMNet is tested on a self-produced UIS extraction dataset and the publicly available UIS-Shenzhen dataset. Quantitative results demonstrate that STMNet achieved the best performance in terms of fa, F 1, and IoU .The mr is slightly higher than that of MAResU-Net and UisNet. Additionally, STMNet achieved the best visual interpretation results and the fastest inference speed on the self-produced UIS extraction dataset