LLM-Based Pose Normalization and Multimodal Fusion for Facial Expression Recognition in Extreme Poses

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Facial expression recognition (FER) technology has progressively matured over time. However, existing FER methods are primarily optimized for frontal face images, and their recognition accuracy significantly degrades when processing profile or large-angle rotated facial images. Consequently, this limitation hinders the practical deployment of FER systems. To mitigate the interference caused by large pose variations and improve recognition accuracy, we propose a FER method based on profile-to-frontal transformation and multimodal learning. Specifically, we first leverage the visual understanding and generation capabilities of Qwen-Image-Edit that transform profile images to frontal viewpoints, preserving key expression features while standardizing facial poses. Second, we introduce the CLIP model to enhance the semantic representation capability of expression features through vision–language joint learning. The qualitative and quantitative experiments on the RAF (89.39%), EXPW (67.17%), and AffectNet-7 (62.66%) datasets demonstrate that our method outperforms the existing approaches.
Funding Information
  • MOE (Ministry of Education in China) Liberal Arts and Social Sciences Foundation (23YJCZH336)
  • Natural Science Foundation of Hubei Province (2025AFC023)
  • Hubei Provincial Department of Education Philosophy and Social Science (23G057)
  • China Postdoctoral Science Foundation General Funding (2025M773206)
  • Hubei Provincial Natural Science Foundation Youth Project (2025AFB140)

This publication has 43 references indexed in Scilit: