Re-Distributing Facial Features for Engagement Prediction with ModernTCN
Open Access
- 1 January 2024
- journal article
- research article
- Published by Tech Science Press in Computers, Materials & Continua
- Vol. 81 (1) , 369-391
Abstract
Automatically detecting learners’ engagement levels helps to develop more effective online teaching and assessment programs, allowing teachers to provide timely feedback and make personalized adjustments based on students’ needs to enhance teaching effectiveness. Traditional approaches mainly rely on single-frame multimodal facial spatial information, neglecting temporal emotional and behavioural features, with accuracy affected by significant pose variations. Additionally, convolutional padding can erode feature maps, affecting feature extraction’s representational capacity. To address these issues, we propose a hybrid neural network architecture, the redistributing facial features and temporal convolutional network (RefEIP). This network consists of three key components: first, utilizing the spatial attention mechanism large kernel attention (LKA) to automatically capture local patches and mitigate the effects of pose variations; second, employing the feature organization and weight distribution (FOWD) module to redistribute feature weights and eliminate the impact of white features and enhancing representation in facial feature maps. Finally, we analyse the temporal changes in video frames through the modern temporal convolutional network (ModernTCN) module to detect engagement levels. We constructed a near-infrared engagement video dataset (NEVD) to better validate the efficiency of the RefEIP network. Through extensive experiments and in-depth studies, we evaluated these methods on the NEVD and the Database for Affect in Situations of Elicitation (DAiSEE), achieving an accuracy of 90.8% on NEVD and 61.2% on DAiSEE in the four-class classification task, indicating significant advantages in addressing engagement video analysis problems.Keywords
This publication has 15 references indexed in Scilit:
- Visual attention networkComputational Visual Media, 2023
- Automatic student engagement measurement using machine learning techniques: A literature study of data and methodsMultimedia Tools and Applications, 2023
- Student-Engagement Detection in Classroom Using Machine Learning AlgorithmElectronics, 2023
- Deep facial spatiotemporal network for engagement prediction in online learningApplied Intelligence, 2021
- Deep face clustering using residual graph convolutional networkKnowledge-Based Systems, 2021
- Deformable face net for pose invariant face recognitionPattern Recognition, 2020
- Advanced, Analytic, Automated (AAA) Measurement of Engagement During LearningEducational Psychologist, 2017
- Joint Face Detection and Alignment Using Multitask Cascaded Convolutional NetworksIEEE Signal Processing Letters, 2016
- 3D face pose estimation by a robust real time tracking of facial featuresMultimedia Tools and Applications, 2014
- The Faces of Engagement: Automatic Recognition of Student Engagementfrom Facial ExpressionsIEEE Transactions on Affective Computing, 2014