Generalizable Facial Expression Recognition
Preprint
Abstract
SOTA facial expression recognition (FER) methods fail on test sets that have domain gaps with the train set. Recent domain adaptation FER methods need to acquire labeled or unlabeled samples of target domains to fine-tune the FER model, which might be infeasible in real-world deployment. In this paper, we aim to improve the zero-shot generalization ability of FER methods on different unseen test sets using only one train set. Inspired by how humans first detect faces and then select expression features, we propose a novel FER pipeline to extract expression-related features from any given face images. Our method is based on the generalizable face features extracted by large models like CLIP. However, it is non-trivial to adapt the general features of CLIP for specific tasks like FER. To preserve the generalization ability of CLIP and the high precision of the FER model, we design a novel approach that learns sigmoid masks based on the fixed CLIP face features to extract expression features. To further improve the generalization ability on unseen test sets, we separate the channels of the learned masked features according to the expression classes to directly generate logits and avoid using the FC layer to reduce overfitting. We also introduce a channel-diverse loss to make the learned masks separated. Extensive experiments on five different FER datasets verify that our method outperforms SOTA FER methods by large margins. Code is available in https://github.com/zyh-uaiaaaa/Generalizable-FER.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Unconstrained Facial Expression Recognition With No-Reference De-Elements LearningIEEE Transactions on Affective Computing, 2023
- Learning Dynamic Relationships for Facial Expression Recognition Based on Graph Convolutional NetworkIEEE Transactions on Image Processing, 2021
- LBAN-IL: A novel method of high discriminative representation for facial expression recognitionNeurocomputing, 2021
- Adaptively Learning Facial Expression Representation via C-F Labels and DistillationIEEE Transactions on Image Processing, 2021
- Correlation-aware adversarial domain adaptation and generalizationPattern Recognition, 2020
- Region Attention Networks for Pose and Occlusion Robust Facial Expression RecognitionIEEE Transactions on Image Processing, 2020
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization2017
- AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the WildIEEE Transactions on Affective Computing, 2017
- Deep Residual Learning for Image Recognition2016
- Facial expression recognition based on Local Binary Patterns: A comprehensive studyImage and Vision Computing, 2009