Boosting Representation Learning for High-Level Semantic Information in Facial Expression Recognition
Open Access
- 8 October 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 22 (10) , 1-22
Abstract
Facial Expression Recognition (FER) requires high-level semantic cues to interpret abstract emotions. However, existing FER approaches commonly suffer from sparse semantic descriptions and unwanted noise from redundant facial pixels. To tackle these issues, we propose a boosting representation learning framework to build a robust high-level semantic projection space connecting textual semantics and facial images. To mitigate semantic scarcity for FER, we develop the Expression High-Level Semantic Encoding (EHSE) module. Guided by FACS-based physiological priors, this module constructs dense semantic prototypes to provide fine-grained supervised information for model learning. To remove irrelevant background noise mixed with redundant facial visual content, the Expression Key-Core Features Focusing (EKFF) module is built, which picks out discriminative facial cues, boosting the signal-to-noise ratio and semantic purity of extracted visual features. To eliminate ambiguous classification boundaries within the semantic projection space, the Expression High-Level Semantic Alignment (EHSA) module is designed, which is equipped with multi-dimensional constraints. Experiments on FER+, RAF-DB, and AffectNet verify the superiority of our approach. Our method obtains state-of-the-art results on FER+ and RAF-DB and achieves the optimal average accuracy over AffectNet’s 7-class and 8-class tasks. These results demonstrate that our framework can construct a balanced, discriminative, and semantically intact projection space for FER.Keywords
Funding Information
- National Natural Science Foundation of China (Grant Nos. 62076086 and 62476077)
- Anhui Provincial Natural Science Foundation (Grant No. 2508085MF150)
- Fundamental Research Funds for the Central Universities (Grant Nos. JZ2024HGTG0308 and PA2023IISL0096)
- Anhui Province Artificial Intelligence Scene Innovation Project (Grant No. 202502501248c09010007)
- Key Laboratory of Knowledge Engineering with Big Data (Grant No. BigKEOpen2025-09)
This publication has 29 references indexed in Scilit:
- A novel deep learning approach for facial emotion recognition: application to detecting emotional responses in elderly individuals with Alzheimer’s diseaseNeural Computing and Applications, 2024
- Facial expression recognition through multi-level features extraction and fusionSoft Computing, 2023
- Facial expression recognition with grid-wise attention and visual transformerInformation Sciences, 2021
- Landmark guidance independent spatio-channel attention and complementary context information based facial expression recognitionPattern Recognition Letters, 2021
- OAENet: Oriented attention ensemble for accurate facial expression recognitionPattern Recognition, 2021
- FER-net: facial expression recognition using deep neural netNeural Computing and Applications, 2021
- Adaptive Feature Mapping for Customizing Deep Learning Based Facial Expression Recognition ModelIEEE Access, 2018
- Challenges in representation learning: A report on three machine learning contestsNeural Networks, 2015
- An argument for basic emotionsCognition and Emotion, 1992
- A circumplex model of affect.Journal of Personality and Social Psychology, 1980