3D Instance Segmentation and Object Detection Framework Based on the Fusion of Lidar Remote Sensing and Optical Image Sensing
Open Access
- 19 August 2021
- journal article
- research article
- Published by MDPI AG in Remote Sensing
- Vol. 13 (16) , 3288
Abstract
Since single sensor and high-density point cloud data processing have certain direct processing limitations in urban traffic scenarios, this paper proposes a 3D instance segmentation and object detection framework for urban transportation scenes based on the fusion of Lidar remote sensing technology and optical image sensing technology. Firstly, multi-source and multi-mode data pre-fusion and alignment of Lidar and camera sensor data are effectively carried out, and then a unique and innovative network of stereo regional proposal selective search-driven DAGNN is constructed. Finally, using the multi-dimensional information interaction, three-dimensional point clouds with multi-features and unique concave-convex geometric characteristics are instance over-segmented and clustered by the hypervoxel storage in the remarkable octree and growing voxels. Finally, the positioning and semantic information of significant 3D object detection in this paper are visualized by multi-dimensional mapping of the boundary box. The experimental results validate the effectiveness of the proposed framework with excellent feedback for small objects, object stacking, and object occlusion. It can be a remediable or alternative plan to a single sensor and provide an essential theoretical and application basis for remote sensing, autonomous driving, environment modeling, autonomous navigation, and path planning under the V2X intelligent network space– ground integration in the future.Keywords
Funding Information
- National Natural Science Foundation of China (61773082)
- Research and Innovation Project for Postgraduate of Chongqing (CYB18173)
- Doctoral Innovative High-end Talents Project (BYJS201808)
- State Scholarship Fund of China Scholarship Council (201908500145)
- Chongqing Technology Innovation and Application Development Special Project (cstc2020jscx-msxmX0153)
This publication has 40 references indexed in Scilit:
- An Improved RANSAC for 3D Point Cloud Plane Segmentation Based on Normal Distribution Transformation CellsRemote Sensing, 2017
- Investigation on the Weighted RANSAC Approaches for Building Roof Plane Segmentation from LiDAR Point CloudsRemote Sensing, 2015
- Scale Space Graph Representation and Kernel Matching for Non Rigid and Textured 3D Shape RetrievalIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Octree-based region growing for point cloud segmentationISPRS Journal of Photogrammetry and Remote Sensing, 2015
- A Segmentation Method for Point Cloud Based on Local Sample and Statistic Inference2015
- Object Partitioning Using Local Convexity2014
- 3D Object Recognition in Cluttered Scenes with Local Surface Features: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Voxel Cloud Connectivity Segmentation - Supervoxels for Point Clouds2013
- A New Weighted Fuzzy C-Means Clustering Algorithm for Remotely Sensed Image ClassificationIEEE Journal of Selected Topics in Signal Processing, 2010
- Curvature Computation on Free-Form 3-D Meshes at Multiple ScalesComputer Vision and Image Understanding, 2001