Automated Analysis of Stereotypical Movements in Videos of Children With Autism Spectrum Disorder
Authors:
Open Access
- 3 September 2024
- journal article
- research article
- Published by American Medical Association (AMA) in JAMA Network Open
- Vol. 7 (9) , e2432851
Abstract
Importance: Stereotypical motor movements (SMMs) are a form of restricted and repetitive behavior, which is a core symptom of autism spectrum disorder (ASD). Current quantification of SMM severity is extremely limited, with studies relying on coarse and subjective caregiver reports or laborious manual annotation of short video recordings.Objective: To assess the utility of a new open-source AI algorithm that can analyze extensive video recordings of children and automatically identify segments with heterogeneous SMMs, thereby enabling their direct and objective quantification.Design, Setting, and Participants: This retrospective cohort study included 241 children (aged 1.4 to 8.0 years) with ASD. Video recordings of 319 behavioral assessments carried out at the Azrieli National Centre for Autism and Neurodevelopment Research in Israel between 2017 and 2021 were extracted. Behavioral assessments included cognitive, language, and autism diagnostic observation schedule, 2nd edition (ADOS-2) assessments. Data were analyzed from October 2020 to May 2024.Exposures: Each assessment was recorded with 2 to 4 cameras, yielding 580 hours of video footage. Within these extensive video recordings, manual annotators identified 7352 video segments containing heterogeneous SMMs performed by different children (21.14 hours of video).Main outcomes and measures: A pose estimation algorithm was used to extract skeletal representations of all individuals in each video frame and was trained an object detection algorithm to identify the child in each video. The skeletal representation of the child was then used to train an SMM recognition algorithm using a 3 dimensional convolutional neural network. Data from 220 children were used for training and data from the remaining 21 children were used for testing.Results: Among 319 behavioral assessment recordings from 241 children (172 [78%] male; mean [SD] age, 3.97 [1.30] years), the algorithm accurately detected 92.53% (95% CI, 81.09%-95.10%) of manually annotated SMMs in our test data with 66.82% (95% CI, 55.28%-72.05%) precision. Overall number and duration of algorithm-identified SMMs per child were highly correlated with manually annotated number and duration of SMMs (r = 0.8; 95% CI, 0.67-0.93; P < .001; and r = 0.88; 95% CI, 0.74-0.96; P < .001, respectively).Conclusions and relevance: This study suggests the ability of an algorithm to identify a highly diverse range of SMMs and quantify them with high accuracy, enabling objective and direct estimation of SMM severity in individual children with ASD.Keywords
This publication has 37 references indexed in Scilit:
- Vision-based activity recognition in children with autism-related behaviorsHeliyon, 2023
- The National Autism Database of Israel: a Resource for Studying Autism Risk Factors, Biomarkers, Outcome Measures, and Treatment EfficacyJournal of Molecular Neuroscience, 2020
- An analysis of stereotypical motor movements and cardiovascular coupling in individuals on the autism spectrumBiological Psychology, 2019
- Brief Report: The Negev Hospital-University-Based (HUB) Autism DatabaseJournal of Autism and Developmental Disorders, 2017
- Do We Need Multiple Informants When Assessing Autistic Traits? The Degree of Report Bias on Offspring, Self, and Spouse RatingsJournal of Autism and Developmental Disorders, 2015
- Restricted and Repetitive Behaviors in Autism Spectrum Disorders and Typical Development: Cross-Sectional and Longitudinal ComparisonsJournal of Autism and Developmental Disorders, 2013
- Automated Detection of Stereotypical Motor MovementsJournal of Autism and Developmental Disorders, 2010
- Social Interaction and Repetitive Motor BehaviorsJournal of Autism and Developmental Disorders, 2007
- Stereotyped Motor Behaviors Associated with Autism in High-risk Infants: A Pilot Videotape Analysis of a Sibling SampleJournal of Autism and Developmental Disorders, 2007
- Repetitive Behavior Profiles in Asperger Syndrome and High-Functioning AutismJournal of Autism and Developmental Disorders, 2005