Meta-MMFNet: Meta-learning-based Multi-model Fusion Network for Micro-expression Recognition
Open Access
- 25 September 2023
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 20 (2) , 1-20
Abstract
Despite its wide applications in criminal investigations and clinical communications with patients suffering from autism, automatic micro-expression recognition remains a challenging problem because of the lack of training data and imbalanced classes problems. In this study, we proposed a meta-learning-based multi-model fusion network (Meta-MMFNet) to solve the existing problems. The proposed method is based on the metric-based meta-learning pipeline, which is specifically designed for few-shot learning and is suitable for model-level fusion. The frame difference and optical flow features were fused, deep features were extracted from the fused feature, and finally in the meta-learning-based framework, weighted sum model fusion method was applied for micro-expression classification. Meta-MMFNet achieved better results than state-of-the-art methods on four datasets. The code is available at https://github.com/wenjgong/meta-fusion-based-method .Keywords
Funding Information
- Spanish Ministry of Economy and Competitiveness
- European Regional Development Fund (PID2020-120311RB-I00/AEI/10.13039/501100011033)
This publication has 51 references indexed in Scilit:
- LBP with Six Intersection Points: Reducing Redundant Information in LBP-TOP for Micro-expression Recognition2015
- A Delaunay-Based Temporal Coding Model for Micro-expression Recognition2015
- CASME II: An Improved Spontaneous Micro-Expression Database and the Baseline EvaluationPLOS ONE, 2014
- The Faces of Engagement: Automatic Recognition of Student Engagementfrom Facial ExpressionsIEEE Transactions on Affective Computing, 2014
- A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind ThemInternational Journal of Computer Vision, 2013
- Eulerian video magnification for revealing subtle changes in the worldACM Transactions on Graphics, 2012
- A Duality Based Approach for Realtime TV-L 1 Optical Flow2007
- How Emotions Work: The Social Functions of Emotional Expression in NegotiationsResearch in Organizational Behavior, 2000
- Learning to Learn: Introduction and Overview1998
- Active Shape Models-Their Training and ApplicationComputer Vision and Image Understanding, 1995