Joint Fusion and Detection via Deep Learning in UAV-Borne Multispectral Sensing of Scatterable Landmine
Open Access
Abstract
Compared with traditional mine detection methods, UAV-based measures are more suitable for the rapid detection of large areas of scatterable landmines, and a multispectral fusion strategy based on a deep learning model is proposed to facilitate mine detection. Using the UAV-borne multispectral cruise platform, we establish a multispectral dataset of scatterable mines, with mine-spreading areas of the ground vegetation considered. In order to achieve the robust detection of occluded landmines, first, we employ an active learning strategy to refine the labeling of the multispectral dataset. Then, we propose an image fusion architecture driven by detection, in which we use YOLOv5 for the detection part, to improve the detection performance instructively while enhancing the quality of the fused image. Specifically, a simple and lightweight fusion network is designed to sufficiently aggregate texture details and semantic information of the source images and obtain a higher fusion speed. Moreover, we leverage detection loss as well as a joint-training algorithm to allow the semantic information to dynamically flow back into the fusion network. Extensive qualitative and quantitative experiments demonstrate that the detection-driven fusion (DDF) that we propose can effectively increase the recall rate, especially for occluded landmines, and verify the feasibility of multispectral data through reasonable processing.Keywords
Funding Information
- National Key Research and Development Program of China (2021YFA0716600, 6142414200607, JCYJ20180307151430655, KQTD20190929172704911)
- Science and Technology on Near-Surface Detection Laboratory Pre-Research Foundation (2021YFA0716600, 6142414200607, JCYJ20180307151430655, KQTD20190929172704911)
- Shenzhen Fundamental Research Program (2021YFA0716600, 6142414200607, JCYJ20180307151430655, KQTD20190929172704911)
- Shenzhen Science and Technology Program (2021YFA0716600, 6142414200607, JCYJ20180307151430655, KQTD20190929172704911)
This publication has 56 references indexed in Scilit:
- Infrared and visible image fusion via gradient transfer and total variation minimizationInformation Fusion, 2016
- Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filtersInformation Fusion, 2016
- Two-scale image fusion of visible and infrared images using saliency detectionInfrared Physics & Technology, 2016
- Fusion of Infrared and Visible Sensor Images Based on Anisotropic Diffusion and Karhunen-Loeve TransformIEEE Sensors Journal, 2015
- A general framework for image fusion based on multi-scale transform and sparse representationInformation Fusion, 2015
- Multispectral pedestrian detection: Benchmark dataset and baseline2015
- Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation2014
- Image fusion based on pixel significance using cross bilateral filterSignal, Image and Video Processing, 2013
- A human perception inspired quality metric for image fusion based on regional informationInformation Fusion, 2007
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004