Abstract
With the increasing use of Unmanned Aerial Vehicles (UAVs), improving the accuracy of semantic segmentation is becoming critical. Depth information preserves geometric structure, serving as an invaluable supplement to color-rich UAV imagery. Inspired by this, we proposed a novel framework named INVITATION, which exclusively takes original UAV imagery as input, yet is capable of obtaining complemented depth information and fusing into RGB semantic segmentation models effectively, thereby enhancing UAV semantic segmentation accuracy. Concretely, this framework supports two distinct depth generation approaches: high-precision Multi-View Stereo (MVS) depth reconstruction using multiple views or video sequences via Structure from Motion (SfM), and monocular depth estimation using individual images. Our empirical evaluations conducted on the UAVid dataset showed that mIoU metric of INVITATION utilized precise reconstructed depth maps via MVS improved from 66.02% to 70.57%, while utilized depth predictions from pre-trained models reached 69.69%, which supports the effectiveness of extracting and fusing depth information from original imagery in enhancing UAV semantic segmentation. This study explores a novel approach to acquire UAV multi-modal information at low data cost, highlights the advantages of incorporating depth information into UAV semantic analysis, and paves the way for further studies on the integration of multi-modal UAV information. Our code is available at https://github.com/CVEO/INVITATION.
Funding Information
  • China Postdoctoral Science Foundation (No. 2020M680109)
  • National Natural Science Foundation of China (No. 42101346)
  • Science and Technology Commission of Shanghai Municipality (No. 22DZ1100800)

This publication has 13 references indexed in Scilit: