Machine and deep learning in facial expression recognition: a survey based on facial action units
Open Access
- 21 June 2026
- journal article
- Published by Springer Nature in Scientific Reports
- Vol. 16 (1)
Abstract
Facial Expression Recognition (FER) has become increasingly relevant in engineering applications spanning security, healthcare, human-robot interaction, and intelligent systems. Accurate recognition of emotional states through facial cues is particularly important for enabling autonomous systems to interpret human intent in dynamic environments. This study presents a survey of recent advances in FER, with a focus on machine learning (ML) and deep learning (DL) methodologies developed between 2019 and 2024. Special attention is given to Facial Action Unit Detection (FAUD) as an intermediate and physiologically grounded representation for expression recognition. The analyzed literature is organized according to the final scope of the proposed models, distinguishing FAUD-oriented, FER-oriented, AU-assisted FER, and FAUD-to-FER approaches. Through this structured comparison, the survey identifies methodological trends, reported performance patterns, and the conditions under which FAUD-based intermediate modeling appears particularly advantageous for reliable FER.Keywords
This publication has 81 references indexed in Scilit:
- Exploratory Analysis of Machine Learning Methodologies Optimization for Facial Expression Recognition (FER)2025
- Facial action units detection using temporal context and feature reassignmentComputer Animation and Virtual Worlds, 2024
- Learning facial expression-aware global-to-local representation for robust action unit detectionApplied Intelligence, 2024
- A comprehensive review of facial expression recognition techniquesMultimedia Systems, 2022
- Stacking multiple cues for facial action unit detectionThe Visual Computer, 2021
- Action unit classification for facial expression recognition using active learning and SVMMultimedia Tools and Applications, 2021
- Experimental Analysis Using Action Units as Feature Descriptor for Emotion in People with down Syndrome2021
- Face Tells Detailed Expression: Generating Comprehensive Facial Expression Sentence Through Facial Action Units2019
- Action Unit Assisted Facial Expression RecognitionLecture Notes in Computer Science, 2019
- Handling missing labels and class imbalance challenges simultaneously for facial action unit recognitionMultimedia Tools and Applications, 2019