Automating Visual Blockage Classification of Culverts with Deep Learning
Open Access
- 18 August 2021
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 11 (16) , 7561
Abstract
Blockage of culverts by transported debris materials is reported as the salient contributor in originating urban flash floods. Conventional hydraulic modeling approaches had no success in addressing the problem primarily because of the unavailability of peak floods hydraulic data and the highly non-linear behavior of debris at the culvert. This article explores a new dimension to investigate the issue by proposing the use of intelligent video analytics (IVA) algorithms for extracting blockage related information. The presented research aims to automate the process of manual visual blockage classification of culverts from a maintenance perspective by remotely applying deep learning models. The potential of using existing convolutional neural network (CNN) algorithms (i.e., DarkNet53, DenseNet121, InceptionResNetV2, InceptionV3, MobileNet, ResNet50, VGG16, EfficientNetB3, NASNet) is investigated over a dataset from three different sources (i.e., images of culvert openings and blockage (ICOB), visual hydrology-lab dataset (VHD), synthetic images of culverts (SIC)) to predict the blockage in a given image. Models were evaluated based on their performance on the test dataset (i.e., accuracy, loss, precision, recall, F1 score, Jaccard Index, region of convergence (ROC) curve), floating point operations per second (FLOPs) and response times to process a single test instance. Furthermore, the performance of deep learning models was benchmarked against conventional machine learning algorithms (i.e., SVM, RF, xgboost). In addition, the idea of classifying deep visual features extracted by CNN models (i.e., ResNet50, MobileNet) using conventional machine learning approaches was also implemented in this article. From the results, NASNet was reported most efficient in classifying the blockage images with the 5-fold accuracy of 85%; however, MobileNet was recommended for the hardware implementation because of its improved response time with 5-fold accuracy comparable to NASNet (i.e., 78%). Comparable performance to standard CNN models was achieved for the case where deep visual features were classified using conventional machine learning approaches. False negative (FN) instances, false positive (FP) instances and CNN layers activation suggested that background noise and oversimplified labelling criteria were two contributing factors in the degraded performance of existing CNN algorithms. A framework for partial automation of the visual blockage classification process was proposed, given that none of the existing models was able to achieve high enough accuracy to completely automate the manual process. In addition, a detection-classification pipeline with higher blockage classification accuracy (i.e., 94%) has been proposed as a potential future direction for practical implementation.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- How computer vision can facilitate flood management: A systematic reviewInternational Journal of Disaster Risk Reduction, 2021
- Computer Vision and IoT-Based Sensors in Flood Monitoring and Mapping: A Systematic ReviewSensors, 2019
- Edge-Computing Video Analytics for Real-Time Traffic Monitoring in a Smart CitySensors, 2019
- Learning Transferable Architectures for Scalable Image Recognition2018
- Design for culvert blockage: the ARR 2016 guidelinesAustralasian Journal of Water Resources, 2018
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Rethinking the Inception Architecture for Computer Vision2016
- Culvert blockages in two Australian flood events and implications for designAustralasian Journal of Water Resources, 2015
- The non-impact of debris blockages on the August 1998 Wollongong floodingAustralasian Journal of Water Resources, 2012
- Hypothesis testing, type I and type II errorsIndustrial Psychiatry Journal, 2009