Micro-Expression Recognition via Fine-Grained Dynamic Perception
Authors:
- Zhiwen ShaoZSZhiwen Shao
- Yifan ChengYCYifan Cheng
- Fan ZhangFZFan Zhang
- Xuehuai ShiXSXuehuai Shi
- Canlin LiCLCanlin Li
- Lizhuang MaLMLizhuang Ma
- Dit-Yan YeungDYDit-Yan Yeung
Preprint
Abstract
Facial micro-expression recognition (MER) is a challenging task, due to the transience, subtlety, and dynamics of micro-expressions (MEs). Most existing methods resort to hand-crafted features or deep networks, in which the former often additionally requires key frames, and the latter suffers from small-scale and low-diversity training data. In this paper, we develop a novel fine-grained dynamic perception (FDP) framework for MER. We propose to rank frame-level features of a sequence of raw frames in chronological order, in which the rank process encodes the dynamic information of both ME appearances and motions. Specifically, a novel local-global feature-aware transformer is proposed for frame representation learning. A rank scorer is further adopted to calculate rank scores of each frame-level feature. Afterwards, the rank features from rank scorer are pooled in temporal dimension to capture dynamic representation. Finally, the dynamic representation is shared by a MER module and a dynamic image construction module, in which the former predicts the ME category, and the latter uses an encoder-decoder structure to construct the dynamic image. The design of dynamic image construction task is beneficial for capturing facial subtle actions associated with MEs and alleviating the data scarcity issue. Extensive experiments show that our method (i) significantly outperforms the state-of-the-art MER methods, and (ii) works well for dynamic image construction. Particularly, our FDP improves by 4.05%, 2.50%, 7.71%, and 2.11% over the previous best results in terms of F1-score on the CASME II, SAMM, CAS(ME)^2, and CAS(ME)^3 datasets, respectively. The code is available at https://github.com/CYF-cuber/FDP.Keywords
All Related Versions
This publication has 33 references indexed in Scilit:
- Global multi-scale extraction and local mixed multi-head attention for facial expression recognition in the wildNeurocomputing, 2025
- Facial Action Unit Detection by Adaptively Constraining Self-Attention and Causally Deconfounding SampleInternational Journal of Computer Vision, 2024
- HTNet for micro-expression recognitionNeurocomputing, 2024
- Empower smart cities with sampling-wise dynamic facial expression recognition via frame-sequence contrastive learningComputer Communications, 2023
- Identity-invariant representation and transformer-style relation for micro-expression recognitionApplied Intelligence, 2023
- A two-stage 3D CNN based learning method for spontaneous micro-expression recognitionNeurocomputing, 2021
- JÂA-Net: Joint Facial Action Unit Detection and Face Alignment Via Adaptive AttentionInternational Journal of Computer Vision, 2020
- OFF-ApexNet on micro-expression recognition systemSignal Processing: Image Communication, 2019
- Less is more: Micro-expression recognition from video using apex frameSignal Processing: Image Communication, 2018
- How Fast are the Leaked Facial Expressions: The Duration of Micro-ExpressionsJournal of Nonverbal Behavior, 2013