Event-Based Gesture and Facial Expression Recognition: A Comparative Analysis
Open Access
- 27 October 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 121269-121283
Abstract
Event-based cameras are novel vision sensors that respond to local variations in intensity, generating asynchronous pixels, referred to as events, with low latency, high temporal resolution, and high dynamic range. These events contain information related to the spatio-temporal dynamics of a scene. Given the temporal nature of the asynchronous event stream, several authors have contributed to recognising deformable objects in motion, specifically gestures. However, another category of deformable objects, such as facial expressions, has yet to be adequately addressed. In this paper, we present a comprehensive review of two topics of interest in novel event-based cameras: gesture and facial expression recognition. For both tasks, we evaluate two existing state-of-the-art learning models, and also we use a third model that learns from temporal and spatial correlations of events. To this end, we evaluate a wide range of classification models across multiple scenarios, analysing: the time/event cut-off window of the sample, the number of samples per class for each database, the spatial resolution of the databases, amongst other factors. In the case of gesture recognition, we utilise existing databases, while in the case of facial expression recognition we have synthetically generated two completely new databases (based on two state-of-the-art image databases): e-CK+ and e-MMI, with promising results for the future of this area. Finally, we provide our contributions to the community, specifically the databases developed and used for this study.Keywords
Funding Information
- Programa de Equipamiento CientÃfico y Tecnológico, ANID, Chile (FONDEQUIP) Project (EQM170041)
This publication has 44 references indexed in Scilit:
- Skimming Digits: Neuromorphic Classification of Spike-Encoded ImagesFrontiers in Neuroscience, 2016
- Direct face detection and video reconstruction from event cameras2016
- Poker-DVS and MNIST-DVS. Their History, How They Were Made, and Other DetailsFrontiers in Neuroscience, 2015
- Converting Static Image Datasets to Spiking Neuromorphic Datasets Using SaccadesFrontiers in Neuroscience, 2015
- A Survey of Face Recognition TechniquesJournal of Information Processing Systems, 2009
- A 128$\times$128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision SensorIEEE Journal of Solid-State Circuits, 2008
- Face recognitionACM Computing Surveys, 2003
- Comprehensive database for facial expression analysis2002
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- A psychoevolutionary theory of emotionsSocial Science Information, 1982