Deep Neural Network based Visual Inspection with 3D Metric Measurement of Concrete Defects using Wall-climbing Robot
Authors:
- 1 November 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2849-2854
Abstract
This paper presents a novel metric inspection robot system using a deep neural network to detect and measure surface flaws (i.e., crack and spalling) on concrete structures performed by a wall-climbing robot. The system consists of four modules: robotics data collection module to obtain RGB-D images and IMU measurement, visual-inertial SLAM module to generate pose coupled key-frames with depth information, InspectionNet module to classify each pixel into three classes (back-ground, crack and spalling), and 3D registration and map fusion module to register the flaw patch into registered 3D model overlaid and highlighted with detected flaws for spatial-contextual visualization. The system enables the metric model of each surface flaw patch with pixel-level accuracy and determines its location in 3D space that is significant for structural health assessment and monitoring. The InspectionNet achieves an average accuracy of 87.64% for crack and spalling inspection. We also demonstrate our InspectionNet is robust to view angle, scale and illumination variation. Finally, we design a metric voxel volume map to highlight the flaw in 3D model and provide location and metric information.Keywords
This publication has 17 references indexed in Scilit:
- Improvement of Crack-Detection Accuracy Using a Novel Crack Defragmentation Technique in Image-Based Road AssessmentJournal of Computing in Civil Engineering, 2016
- A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructureAdvanced Engineering Informatics, 2015
- Automated Crack Detection on Concrete BridgesIEEE Transactions on Automation Science and Engineering, 2014
- Visual inspection strategies for large bridges using Unmanned Aerial Vehicles (UAV)2014
- Long-distance precision inspection method for bridge cracks with image processingAutomation in Construction, 2014
- Contour Detection and Hierarchical Image SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010
- Bridge inspection robot system with machine visionAutomation in Construction, 2009
- Fast Ego-motion Estimation with Multi-rate Fusion of Inertial and VisionThe International Journal of Robotics Research, 2007
- A Multi-State Constraint Kalman Filter for Vision-aided Inertial NavigationProceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (cat. No.01ch37164), 2007
- Fast dynamic grid deformation based on Delaunay graph mappingJournal of Computational Physics, 2006