Image Annotation by Eye Tracking: Accuracy and Precision of Centerlines of Obstructed Small-Bowel Segments Placed Using Eye Trackers
- 29 May 2019
- journal article
- research article
- Published by Springer Nature in Journal of Digital Imaging
- Vol. 32 (5) , 855-864
Abstract
Small-bowel obstruction (SBO) is a common and important disease, for which machine learning tools have yet to be developed. Image annotation is a critical first step for development of such tools. This study assesses whether image annotation by eye tracking is sufficiently accurate and precise to serve as a first step in the development of machine learning tools for detection of SBO on CT. Seven subjects diagnosed with SBO by CT were included in the study. For each subject, an obstructed segment of bowel was chosen. Three observers annotated the centerline of the segment by manual fiducial placement and by visual fiducial placement using a Tobii 4c eye tracker. Each annotation was repeated three times. The distance between centerlines was calculated after alignment using dynamic time warping (DTW) and statistically compared to clinical thresholds for diagnosis of SBO. Intra-observer DTW distance between manual and visual centerlines was calculated as a measure of accuracy. These distances were 1.1 ± 0.2, 1.3 ± 0.4, and 1.8 ± 0.2 cm for the three observers and were less than 1.5 cm for two of three observers (P < 0.01). Intra- and inter-observer DTW distances between centerlines placed with each method were calculated as measures of precision. These distances were 0.6 ± 0.1 and 0.8 ± 0.2 cm for manual centerlines, 1.1 ± 0.4 and 1.9 ± 0.6 cm for visual centerlines, and were less than 3.0 cm in all cases (P < 0.01). Results suggest that eye tracking–based annotation is sufficiently accurate and precise for small-bowel centerline annotation for use in machine learning–based applications.Keywords
Funding Information
- National Institutes of Health (T32EB005970)
This publication has 37 references indexed in Scilit:
- Random forest active learning for AAA thrombus segmentation in computed tomography angiography imagesNeurocomputing, 2014
- Machine learning and radiologyMedical Image Analysis, 2012
- Accuracy of a Remote Eye Tracker for Radiologic Observer Studies: Effects of Calibration and Recording EnvironmentAcademic Radiology, 2012
- Neural network based focal liver lesion diagnosis using ultrasound imagesComputerized Medical Imaging and Graphics, 2011
- HopDSW: An approximate dynamic space warping algorithm for fast shape matching and retrievalJournal of King Saud University Computer and Information Sciences, 2011
- Whole Brain SegmentationNeuron, 2002
- Cortical Surface-Based AnalysisNeuroImage, 1999
- Multiscale vessel enhancement filtering1998
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- CT of small-bowel obstruction: value in establishing the diagnosis and determining the degree and cause.American Journal of Roentgenology, 1994