Automated 2D, 2.5D, and 3D Segmentation of Coral Reef Pointclouds and Orthoprojections
Authors:
Open Access
- 27 May 2022
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Robotics and AI
- Vol. 9, 884317
Abstract
Enabled by advancing technology, coral reef researchers increasingly prefer use of image-based surveys over approaches depending solely upon in situ observations, interpretations, and recordings of divers. The images collected, and derivative products such as orthographic projections and 3D models, allow researchers to study a comprehensive digital twin of their field sites. Spatio-temporally located twins can be compared and annotated, enabling researchers to virtually return to sites long after they have left them. While these new data expand the variety and specificity of biological investigation that can be pursued, they have introduced the much-discussed Big Data Problem: research labs lack the human and computational resources required to process and analyze imagery at the rate it can be collected. The rapid development of unmanned underwater vehicles suggests researchers will soon have access to an even greater volume of imagery and other sensor measurements than can be collected by diver-piloted platforms, further exacerbating data handling limitations. Thoroughly segmenting (tracing the extent of and taxonomically identifying) organisms enables researchers to extract the information image products contain, but is very time-consuming. Analytic techniques driven by neural networks offer the possibility that the segmentation process can be greatly accelerated through automation. In this study, we examine the efficacy of automated segmentation on three different image-derived data products: 3D models, and 2D and 2.5D orthographic projections thereof; we also contrast their relative accessibility and utility to different avenues of biological inquiry. The variety of network architectures and parameters tested performed similarly, ∼80% IoU for the genus Porites, suggesting that the primary limitations to an automated workflow are 1) the current capabilities of neural network technology, and 2) consistency and quality control in image product collection and human training/testing dataset generation.Keywords
This publication has 19 references indexed in Scilit:
- 3D Semantic Segmentation with Submanifold Sparse Convolutional Networks2018
- 3D photogrammetry quantifies growth and external erosion of individual coral colonies and skeletonsScientific Reports, 2017
- Large-area imaging reveals biologically driven non-random spatial patterns of corals at a remote reefCoral Reefs, 2017
- Ror2 signaling regulates Golgi structure and transport through IFT20 for tumor invasivenessScientific Reports, 2017
- Re-evaluating the health of coral reef communities: baselines and evidence for human impacts across the central PacificProceedings Of The Royal Society B-Biological Sciences, 2016
- ‘Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applicationsGeomorphology, 2012
- Terrestrial ecosystems from space: a review of earth observation products for macroecology applicationsGlobal Ecology and Biogeography, 2011
- Large Area 3-D Reconstructions From Underwater Optical SurveysIEEE Journal of Oceanic Engineering, 2009
- Underwater Video Mosaics as Visual Navigation MapsComputer Vision and Image Understanding, 2000
- Profile Report: ASHA Member SurveyJournal of School Health, 1981