Lightweight Cross-Modal Information Measure and Propagation for Road Extraction From Remote Sensing Image and Trajectory/LiDAR
- 5 April 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 62 (01962892) , 1-16
Abstract
Recent studies have confirmed that GPS trajectory can effectively assist in achieving more accurate road extraction from remote sensing images. Therefore, lots of efforts focus on designing effective multi-modal fusion strategies for GPS trajectory and remote sensing image modalities. However, there are still some limitations, e.g ., the fusion structures are complex and hinder further improvements. Moreover, the negative impact of redundant information in various modalities is commonly ignored. This paper aims to design a simple yet effective fusion strategy for GPS trajectory and remote sensing image modalities to address the above issues. Inspired by the network pruning algorithm, we design a Cross-Modal Information Propagation (CMIP) mechanism. CMIP utilizes the scaling factors and sparse constraint to distinguish the redundant information of a certain modality that is directly replaced with corresponding information of another modality. We improve the widely-used L1 sparse constraint and propose a novel information balanced constraint which is added on the scaling factors to better identify and prune redundant channels. Embedded in the CMIP mechanism, a multi-modal information propagation network (CMIPNet) is proposed, which can fully explore the complementarities between different modalities to accurately locate roads, especially roads with noise or incomplete information of a certain modality. Since the CMIP is parameter-free and self-adaptive, CMIPNet is lightweight and easy to deploy. The parameter number of CMIPNet can be comparable to single-modal models, which is about 1/3 of the existing multi-modal models. Extensive experiments are performed to demonstrate that CMIPNet outperforms the previous single- and multi-modal road extraction methods.Keywords
Funding Information
- China Postdoctoral Science Foundation (2023M731803, 2023M741126)
- China National Postdoctoral Program for Innovative Talents (BX20230108)
- National Natural Science Foundation of China (62303173)
This publication has 57 references indexed in Scilit:
- Cross-Modal Transformer for RGB-D semantic segmentation of production workshop objectsPattern Recognition, 2023
- Delivering Arbitrary-Modal Semantic Segmentation2023
- Leveraging Crowdsourced GPS Data for Road Extraction From Aerial Imagery2019
- D-LinkNet: LinkNet with Pretrained Encoder and Dilated Convolution for High Resolution Satellite Imagery Road Extraction2018
- ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices2018
- Road centerline extraction from airborne LiDAR point cloud based on hierarchical fusion and optimizationISPRS Journal of Photogrammetry and Remote Sensing, 2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Deep learningNature, 2015
- Water flow based geometric active deformable model for road networkISPRS Journal of Photogrammetry and Remote Sensing, 2015
- Automatic extraction of urban road networks from multi-view aerial imageryISPRS Journal of Photogrammetry and Remote Sensing, 2003