Automatic action unit detection in infants using convolutional neural network
Authors:
- 1 October 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 2017, 216-221
Abstract
Action unit detection in infants relative to adults presents unique challenges. Jaw contour is less distinct, facial texture is reduced, and rapid and unusual facial movements are common. To detect facial action units in spontaneous behavior of infants, we propose a multi-label Convolutional Neural Network (CNN). Eighty-six infants were recorded during tasks intended to elicit enjoyment and frustration. Using an extension of FACS for infants (Baby FACS), over 230,000 frames were manually coded for ground truth. To control for chance agreement, inter-observer agreement between Baby-FACS coders was quantified using free-margin kappa. Kappa coefficients ranged from 0.79 to 0.93, which represents high agreement. The multi-label CNN achieved comparable agreement with manual coding. Kappa ranged from 0.69 to 0.93. Importantly, the CNN-based AU detection revealed the same change in findings with respect to infant expressiveness between tasks. While further research is needed, these findings suggest that automatic AU detection in infants is a viable alternative to manual coding of infant facial expression.Keywords
This publication has 39 references indexed in Scilit:
- Darwin’s Duchenne: Eye Constriction during Infant Joy and DistressPLOS ONE, 2013
- Real-time 2D+3D facial action and expression recognitionPattern Recognition, 2010
- Automatic Recognition of Facial Actions in Spontaneous ExpressionsJournal of Multimedia, 2006
- Active Affective State Detection and User Assistance With Dynamic Bayesian NetworksIEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 2004
- Recognizing action units for facial expression analysisIEEE Transactions on Pattern Analysis and Machine Intelligence, 2001
- Detection, tracking, and classification of action units in facial expressionRobotics and Autonomous Systems, 2000
- Production of emotional facial expressions in European American, Japanese, and Chinese infants.Developmental Psychology, 1998
- Are max-specified infant facial expressions during face-to-face interaction consistent with differential emotions theory?Developmental Psychology, 1993
- Infant response in the still-face paradigm at 6 months predicts avoidant and secure attachment at 12 monthsDevelopment and Psychopathology, 1991
- Coefficient Kappa: Some Uses, Misuses, and AlternativesEducational and Psychological Measurement, 1981