Ensemble Spatial and Temporal Vision Transformer for Action Units Detection
- 1 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5770-5776
Abstract
Facial Action Units detection (FAUs) represents a fine-grained classification problem that involves identifying different units on the human face, as defined by the Facial Action Coding System. In this paper, we present a simple yet efficient Vision Transformer-based approach for addressing the task of Action Units (AU) detection in the context of Affective Behavior Analysis in-the-wild (ABAW) competition. We employ the Video Vision Transformer(ViViT) Network to capture the temporal facial change in the video. Besides, to reduce massive size of the Vision Transformers model, we replace the ViViT feature extraction layers with the CNN backbone (Regnet). Our model outperform the baseline model of ABAW 2023 challenge [8], with a notable 14% difference in result. Our team has achieved a position within the top 5 teams in the ABAW 2023 competition, scoring slightly below the top three and four teams by a narrow margin of 0.27% and 0.43%, respectively.Keywords
Funding Information
- National Research Foundation
- National Research Foundation
- Ministry of Education
This publication has 17 references indexed in Scilit:
- Designing Network Design Spaces2020
- Deep Affect Prediction in-the-Wild: Aff-Wild Database and Challenge, Deep Architectures, and BeyondInternational Journal of Computer Vision, 2019
- Focal Loss for Dense Object Detection2017
- Automatic Analysis of Facial Actions: A SurveyIEEE Transactions on Affective Computing, 2017
- Aff-Wild: Valence and Arousal ‘In-the-Wild’ Challenge2017
- EAC-Net: A Region-Based Deep Enhancing and Cropping Approach for Facial Action Unit Detection2017
- BP4D-Spontaneous: a high-resolution spontaneous 3D dynamic facial expression databaseImage and Vision Computing, 2014
- DISFA: A Spontaneous Facial Action Intensity DatabaseIEEE Transactions on Affective Computing, 2013
- ImageNet: A large-scale hierarchical image database2009
- Emotion recognition in human-computer interactionIEEE Signal Processing Magazine, 2001