Abstract
Facial Expression Recognition (FER) requires high-level semantic cues to interpret abstract emotions. However, existing FER approaches commonly suffer from sparse semantic descriptions and unwanted noise from redundant facial pixels. To tackle these issues, we propose a boosting representation learning framework to build a robust high-level semantic projection space connecting textual semantics and facial images. To mitigate semantic scarcity for FER, we develop the Expression High-Level Semantic Encoding (EHSE) module. Guided by FACS-based physiological priors, this module constructs dense semantic prototypes to provide fine-grained supervised information for model learning. To remove irrelevant background noise mixed with redundant facial visual content, the Expression Key-Core Features Focusing (EKFF) module is built, which picks out discriminative facial cues, boosting the signal-to-noise ratio and semantic purity of extracted visual features. To eliminate ambiguous classification boundaries within the semantic projection space, the Expression High-Level Semantic Alignment (EHSA) module is designed, which is equipped with multi-dimensional constraints. Experiments on FER+, RAF-DB, and AffectNet verify the superiority of our approach. Our method obtains state-of-the-art results on FER+ and RAF-DB and achieves the optimal average accuracy over AffectNet’s 7-class and 8-class tasks. These results demonstrate that our framework can construct a balanced, discriminative, and semantically intact projection space for FER.
Funding Information
  • National Natural Science Foundation of China (Grant Nos. 62076086 and 62476077)
  • Anhui Provincial Natural Science Foundation (Grant No. 2508085MF150)
  • Fundamental Research Funds for the Central Universities (Grant Nos. JZ2024HGTG0308 and PA2023IISL0096)
  • Anhui Province Artificial Intelligence Scene Innovation Project (Grant No. 202502501248c09010007)
  • Key Laboratory of Knowledge Engineering with Big Data (Grant No. BigKEOpen2025-09)

This publication has 29 references indexed in Scilit: