On the Effectiveness of Weakly Supervised Semantic Segmentation for Building Extraction From High-Resolution Remote Sensing Imagery
Open Access
- 4 March 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 14 (19391404) , 3266-3281
Abstract
A critical obstacle to achieve semantic segmentation of remote sensing images by the deep convolutional neural network is the requirement of huge pixel-level labels. Taking building extraction as an example, this study focuses on how to effectively apply weakly supervised semantic segmentation (WSSS) to high-resolution remote sensing (HR) images with image-level labels, which is a prominent solution for the huge labeling challenge. The widely used two-step WSSS framework is adopted, in which the pseudo-masks are first produced from image-level labels and followed by a segmentation network trained by the pseudo-masks. In addition, the fully connected conditional random field (CRF) is utilized to explore spatial context in both training and prediction stages. Detailed analyzes are implemented on applying WSSS on HR images in terms of producing pseudo-masks, training segmentation network, and optimizing predictions. We show that the tradeoff between precision and recall of pseudo-masks, as well as the boundary accuracy and the background, needs to be carefully considered. The benefits of the segmentation network in the two-step framework are demonstrated in comparison to using classification network only for WSSS, and the effects of CRF-loss are identified to be powerful for improving the segmentation network while it is not appropriate for dense buildings. An overlapping strategy and CRF postprocessing are further demonstrated to be effective for optimizing the segmentation results during inferencing. Through deliberate settings, we can generate results comparable to fully supervised on the ISPRS Potsdam and Vaihingen dataset, which is meaningful for promoting WSSS applications for extracting geographic information from HR images.Keywords
Funding Information
- National Natural Science Foundation of China (42071297, 41871235)
- Fundamental Research Funds for the Central Universities (020914380080)
- Jiangsu Province of China
This publication has 52 references indexed in Scilit:
- Automatic Road Detection and Centerline Extraction via Cascaded End-to-End Convolutional Neural NetworkIEEE Transactions on Geoscience and Remote Sensing, 2017
- Convolutional Neural Networks for Large-Scale Remote-Sensing Image ClassificationIEEE Transactions on Geoscience and Remote Sensing, 2016
- Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation2016
- Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the ArtIEEE Geoscience and Remote Sensing Magazine, 2016
- Deep Residual Learning for Image Recognition2016
- Learning Deep Features for Discriminative Localization2016
- Semantic Annotation of High-Resolution Satellite Images via Weakly Supervised LearningIEEE Transactions on Geoscience and Remote Sensing, 2016
- Semantic segmentation of urban scenes by learning local class interactions2015
- Fully convolutional networks for semantic segmentation2015
- Geographic Object-Based Image Analysis – Towards a new paradigmISPRS Journal of Photogrammetry and Remote Sensing, 2014