Deep Learning–Based Precision Cropping of Eye Regions in Strabismus Photographs: Algorithm Development and Validation Study for Workflow Optimization
Open Access
- 17 July 2025
- journal article
- research article
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 27, e74402
Abstract
Background: Traditional ocular gaze photograph preprocessing, relying on manual cropping and head tilt correction, is time-consuming and inconsistent, limiting artificial intelligence (AI) model development and clinical application. Objective: This study aimed to address these challenges using an advanced preprocessing algorithm to enhance the accuracy, efficiency, and standardization of eye region cropping for clinical workflows and AI data preprocessing. Methods: This retrospective and prospective cross-sectional study utilized 5832 images from 648 inpatients and outpatients, capturing 3 gaze positions under diverse conditions, including obstructions and varying distances. The preprocessing algorithm, based on a rotating bounding box detection framework, was trained and evaluated using precision, recall, and mean average precision (mAP) at various intersections over union thresholds. A 5-fold cross-validation was performed on an inpatient dataset, with additional testing on an independent outpatient dataset and an external cross-population dataset of 500 images from the IMDB-WIKI collection, representing diverse ethnicities and ages. Expert validation confirmed alignment with clinical standards across 96 images (48 images from a Chinese dataset of patients with strabismus and 48 images from IMDB-WIKI). Gradient-weighted class activation mapping heatmaps were used to assess model interpretability. A control experiment with 5 optometry specialists compared manual and automated cropping efficiency. Downstream task validation involved preprocessing 1000 primary gaze photographs using the Dlib toolkit, faster region-based convolutional neural network (R-CNN; both without head tilt correction), and our model (with correction), evaluating the impact of head tilt correction via the vision transformer strabismus screening network through 5-fold cross-validation. Results: The model achieved exceptional performance across datasets: on the 5-fold cross-validation set, it recorded a mean precision of 1.000 (95% CI 1.000‐1.000), recall of 1.000 (95% CI 1.000‐1.000), mAP50 of 0.995 (95% CI 0.995‐0.995), and mAP95 of 0.893 (95% CI 0.870‐0.918); on the internal independent test set, precision and recall were 1.000, with mAP50 of 0.995 and mAP95 of 0.801; and on the external cross-population test set, precision and recall were 1.000, with mAP50 of 0.937 and mAP95 of 0.792. The control experiment reduced image preparation time from 10 hours for manual cropping of 900 photos to 30 seconds with the automated model. Downstream strabismus screening task validation showed our model (with head tilt correction) achieving an area under the curve of 0.917 (95% CI 0.901‐0.933), surpassing Dlib-toolkit and faster R-CNN (both without head tilt correction) with an area under the curve of 0.856 (P=.02) and 0.884 (P=.05), respectively. Heatmaps highlighted core ocular focus, aligning with head tilt directions. Conclusions: This study delivers an AI-driven platform featuring a preprocessing algorithm that automates eye region cropping, correcting head tilt variations to improve image quality for AI development and clinical use. Integrated with electronic archives and patient-physician interaction, it enhances workflow efficiency, ensures telemedicine privacy, and supports ophthalmological research and strabismus care.Keywords
This publication has 19 references indexed in Scilit:
- An artificial intelligence platform for the screening and managing of strabismusEye, 2024
- Comparison of RetinaNet, SSD, and YOLO v3 for real-time pill identificationBMC Medical Informatics and Decision Making, 2021
- Detection of Referable Horizontal Strabismus in Children's Primary Gaze Photographs Using Deep LearningTranslational Vision Science & Technology, 2021
- Automatic Detection of Melanoma with Yolo Deep Convolutional Neural Networks2019
- Mask R-CNN2017
- SSD: Single Shot MultiBox Detector2016
- You Only Look Once: Unified, Real-Time Object Detection2016
- One millisecond face alignment with an ensemble of regression trees2014
- Comitant horizontal strabismus: an Asian perspectiveBritish Journal of Ophthalmology, 2007
- Impact of large angle horizontal strabismus on ability to obtain employmentOphthalmology, 2000