Facial Expression Recognition in Educational Research From the Perspective of Machine Learning: A Systematic Review
Open Access
- 6 October 2023
- journal article
- review article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 112060-112074
Abstract
Facial expression analysis aims to understand human emotions by analyzing visual face information and is a popular topic in the computer vision community. In educational research, the analyzed students’ affect states can be used by faculty members as feedback to improve their teaching style and strategy so that the learning rate of all the students present can be enhanced. Facial expression analysis has attracted much attention in educational research, and a few reviews on this topic have emerged. However, previous reviews on facial expression recognition methods in educational research focus mostly on summarizing the existing literature on emotion models from a theoretical perspective, neglecting technical summaries of facial expression recognition. In order to advance the development of facial expression analysis in educational research, this paper outlines the tasks, progress, challenges, and future trends related to facial expression analysis. First, facial expression recognition methods in educational research lack an overall framework. Second, studies based on the latest machine learning methods are not mentioned in previous reviews. Finally, some key challenges have not been fully explored. Therefore, unlike previous reviews, this systematic review summarizes two kinds of educational research methods based on facial expression recognition and their application scenarios. Then, an overall framework is proposed, along with various kinds of machine learning methods and published datasets. Finally, the key challenges of face occlusion and the expression uncertainty problem are presented. This study aims to capture the full picture of facial expression recognition methods in educational research from a machine learning perspective.Keywords
Funding Information
- China Postdoctoral Science Foundation (2021M692006)
- National Natural Science Foundation of China (62107027, 62177032)
This publication has 92 references indexed in Scilit:
- Towards multimodal emotion recognition in e-learning environmentsInteractive Learning Environments, 2014
- The Faces of Engagement: Automatic Recognition of Student Engagementfrom Facial ExpressionsIEEE Transactions on Affective Computing, 2014
- The Definition, Assessment, and Mitigation of State Boredom Within Educational Settings: A Comprehensive ReviewEducational Psychology Review, 2011
- Theories, methods and current research on emotions in library and information science, information retrieval and human–computer interactionInformation Processing & Management, 2011
- Measuring emotions in students’ learning and performance: The Achievement Emotions Questionnaire (AEQ)Contemporary Educational Psychology, 2011
- A longitudinal analysis of achievement goals: From affective antecedents to emotional effects and achievement outcomes.Journal of Educational Psychology, 2009
- Facial expression recognition based on Local Binary Patterns: A comprehensive studyImage and Vision Computing, 2009
- Achievement goals and achievement emotions: Testing a model of their joint relations with academic performance.Journal of Educational Psychology, 2009
- An analysis of facial expression recognition under partial facial image occlusionImage and Vision Computing, 2008
- Automatic detection of learner’s affect from conversational cuesUser Modeling and User-Adapted Interaction, 2007