A Spatio-Temporal Attention-Based Model for Infant Movement Assessment From Videos
- 6 May 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 25 (10) , 3911-3920
Abstract
The absence or abnormality of fidgety movements of joints or limbs is strongly indicative of cerebral palsy in infants. Developing computer-based methods for assessing infant movements in videos is pivotal for improved cerebral palsy screening. Most existing methods use appearance-based features and are thus sensitive to strong but irrelevant signals caused by background clutter or a moving camera. Moreover, these features are computed over the whole frame, thus they measure gross whole body movements rather than specific joint/limb motion. Addressing these challenges, we develop and validate a new method for fidgety movement assessment from consumer-grade videos using human poses extracted from short clips. Human poses capture only relevant motion profiles of joints and limbs and are thus free from irrelevant appearance artifacts. The dynamics and coordination between joints are modeled using spatio-temporal graph convolutional networks. Frames and body parts that contain discriminative information about fidgety movements are selected through a spatio-temporal attention mechanism. We validate the proposed model on the cerebral palsy screening task using a real-life consumer-grade video dataset collected at an Australian hospital through the Cerebral Palsy Alliance, Australia. Our experiments show that the proposed method achieves the ROC-AUC score of 81.87%, significantly outperforming existing competing methods with better interpretability.Keywords
All Related Versions
Funding Information
- Cerebral Palsy Alliance Research Foundation
- Mater Misericordiae Ltd
- Human Research Ethics Committee (EC00332, HREC/14/MHS/188)
This publication has 27 references indexed in Scilit:
- Are sporadic fidgety movements as clinically relevant as is their absence?Early Human Development, 2015
- Jerky spontaneous movements at term age in preterm infants who later developed cerebral palsyEarly Human Development, 2014
- Computer‐based analysis of general movements reveals stereotypies predicting cerebral palsyDevelopmental Medicine & Child Neurology, 2014
- Early Markers For Cerebral Palsy: Insights From the Assessment of General MovementsFuture Neurology, 2012
- Kinematic assessment of stereotypy in spontaneous movements in infantsGait & Posture, 2012
- Early prediction of cerebral palsy by computer‐based video analysis of general movements: a feasibility studyDevelopmental Medicine & Child Neurology, 2010
- Movement analysis by accelerometry of newborns and infants for the early detection of movement disorders due to infantile cerebral palsyMedical & Biological Engineering & Computing, 2010
- Using computer-based video analysis in the study of fidgety movementsEarly Human Development, 2009
- Quantification of the segmental kinematics of spontaneous infant movementsJournal of Biomechanics, 2008
- Long Short-Term MemoryNeural Computation, 1997