Learning from Machine Learning: advancing from static to dynamic facial function quantification
Preprint
Abstract
Objectives We assess an open-source Python machine learning algorithm’s efficacy in image and video analysis of facial palsy (FP) patients. Methods Images and videos of 60 patients with varying FP severities performing standard movements were obtained from MEEI Facial Palsy database. Landmarks generated on images by the open-source algorithm (adapted from OpenCV and Dlib libraries) and Emotrics (standard for two-dimensional FP analysis) were compared. Considering the human eye as the standard for accuracy, three raters marked perceived errors in each algorithm’s tracking of five facial features. Cumulative error distributions between both algorithms were compared via normalized root mean square error. FP severity and facial feature-specific error rates were compared using ANOVA tests. Raters also analyzed open-source algorithm-generated video landmarks; similar statistical comparisons between open-source algorithms’ image and video-based analyses were performed. Results Cumulative error distribution between both algorithms’ image analyses was most robust for normal function; significant discrepancies were observed in mild/moderate flaccidity and nearly-normal/complete synkinesis. Both algorithms had similar error rates across all facial features (p=0.76) and FP severities (p=0.37). In the open-source algorithm’s video analysis, mild synkinesis (24.7%) and severe flaccidity (19.7%) had the highest error rates. Comparing image and video analyses generated by the open-source algorithm, video analyses had lower error rates across all FP severities (pConclusions We report for the first time the feasibility and relative accuracy of a Python open-source algorithm for dynamic facial landmark tracking in FP videos. The demonstrated superiority of landmark tracking with videos over images can improve objective FP quantification.Keywords
This publication has 27 references indexed in Scilit:
- Quality of Life in Bell's Palsy: Correlation with Sunnybrook and House‐Brackmann Over TimeThe Laryngoscope, 2020
- Spontaneity Assessment in Dually Innervated Gracilis Smile Reanimation SurgeryJAMA Facial Plastic Surgery, 2019
- Clinician and Automated Assessments of Facial Function Following Eyelid Weight PlacementJAMA Facial Plastic Surgery, 2019
- The spectrum of facial palsy: The MEEI facial palsy photo and video standard setThe Laryngoscope, 2019
- Use of Objective Metrics in Dynamic Facial ReanimationJAMA Facial Plastic Surgery, 2018
- Interobserver variability of the House-Brackmann facial nerve grading system for the analysis of a randomized multi-center phase III trialActa Neurochirurgica, 2017
- Emerging vs Time-Tested Methods of Facial Grading Among Patients With Facial ParalysisJAMA Facial Plastic Surgery, 2016
- Standard Outcome Measures in Facial ParalysisJAMA Facial Plastic Surgery, 2016
- The use of the facial clinimetric evaluation scale as a patient‐based grading system in bell's palsyThe Laryngoscope, 2013
- Tip Points: Defining the TipAesthetic Plastic Surgery, 1999