Terrain-Informed Self-Supervised Learning: Enhancing Building Footprint Extraction from LiDAR Data with Limited Annotations
Preprint
Abstract
Estimating building footprint maps from geospatial data is of paramount importance in urban planning, development, disaster management, and various other applications. Deep learning methodologies have gained prominence in building segmentation maps, offering the promise of precise footprint extraction without extensive post-processing. However, these methods face challenges in generalization and label efficiency, particularly in remote sensing, where obtaining accurate labels can be both expensive and time-consuming. To address these challenges, we propose terrain-aware self-supervised learning, tailored to remote sensing, using digital elevation models from LiDAR data. We propose to learn a model to differentiate between bare Earth and superimposed structures enabling the network to implicitly learn domain-relevant features without the need for extensive pixel-level annotations. We test the effectiveness of our approach by evaluating building segmentation performance on test datasets with varying label fractions. Remarkably, with only 1% of the labels (equivalent to 25 labeled examples), our method improves over ImageNet pre-training, showing the advantage of leveraging unlabeled data for feature extraction in the domain of remote sensing. The performance improvement is more pronounced in few-shot scenarios and gradually closes the gap with ImageNet pre-training as the label fraction increases. We test on a dataset characterized by substantial distribution shifts and labeling errors to demonstrate the generalizability of our approach. When compared to other baselines, including ImageNet pretraining and more complex architectures, our approach consistently performs better, demonstrating the efficiency and effectiveness of self-supervised terrain-aware feature learning.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Automatic 3D building reconstruction from multi-view aerial images with deep learningISPRS Journal of Photogrammetry and Remote Sensing, 2021
- Remote Sensing Image Scene Classification With Self-Supervised Paradigm Under Limited Labeled SamplesIEEE Geoscience and Remote Sensing Letters, 2020
- Fully Convolutional Networks for Multisource Building Extraction From an Open Aerial and Satellite Imagery Data SetIEEE Transactions on Geoscience and Remote Sensing, 2018
- Automatic Building Segmentation of Aerial Imagery Using Multi-Constraint Fully Convolutional NetworksRemote Sensing, 2018
- Deep Learning in Remote Sensing: A Comprehensive Review and List of ResourcesIEEE Geoscience and Remote Sensing Magazine, 2017
- Learning Building Extraction in Aerial Scenes with Convolutional NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- A general-purpose framework for parallel processing of large-scale LiDAR dataInternational Journal of Digital Earth, 2017
- Convolutional Neural Networks for Large-Scale Remote-Sensing Image ClassificationIEEE Transactions on Geoscience and Remote Sensing, 2016
- Data fusion of high-resolution satellite imagery and LiDAR data for automatic building extractionISPRS Journal of Photogrammetry and Remote Sensing, 2007
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004