A Multimodal Driver Anger Recognition Method Based on Context-Awareness
Open Access
- 3 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 118533-118550
Abstract
In today’s society, the harm of driving anger to traffic safety is increasingly prominent. With the development of human-computer interaction and intelligent transportation systems, the application of biometric technology in driver emotion recognition has attracted widespread attention. This study proposes a context-aware multi-modal driver anger emotion recognition method (CA-MDER) to address the main issues encountered in multi-modal emotion recognition tasks. These include individual differences among drivers, variability in emotional expression across different driving scenarios, and the inability to capture driving behavior information that represents vehicle-to-vehicle interaction. The method employs Attention Mechanism-Depthwise Separable Convolutional Neural Networks (AM-DSCNN), an improved Support Vector Machines (SVM), and Random Forest (RF) models to perform multi-modal anger emotion recognition using facial, vocal, and driving state information. It also uses Context-Aware Reinforcement Learning (CA-RL) based adaptive weight distribution for multi-modal decision-level fusion. The results show that the proposed method performs well in emotion classification metrics, with an accuracy and F1 score of 91.68% and 90.37%, respectively, demonstrating robust multi-modal emotion recognition performance and powerful emotion recognition capabilities.Keywords
Funding Information
- National Key Research and Development Program Project “Major Accident Risk Prevention and Emergency Avoidance Technology for Road Transport Vehicles” (2023YFC3009600)
- Graduate Innovation Fund of Jilin University (2023YFC3009600)
This publication has 46 references indexed in Scilit:
- Hierarchical multimodal-fusion of physiological signals for emotion recognition with scenario adaption and contrastive alignmentInformation Fusion, 2023
- Integration of Driver Behavior into Emotion Recognition Systems: A Preliminary Study on Steering Wheel and Vehicle Acceleration2019
- Facial expression recognition using distance and texture signature relevant featuresApplied Soft Computing, 2019
- Adaptive facial point detection and emotion recognition for a humanoid robotComputer Vision and Image Understanding, 2015
- Facial emotion recognition using empirical mode decompositionExpert Systems with Applications, 2015
- Focusing on appraisals: How and why anger and fear influence driving risk perceptionJournal of Safety Research, 2013
- Facial expression recognition from near-infrared videosImage and Vision Computing, 2011
- Spoken emotion recognition using hierarchical classifiersComputer Speech & Language, 2011
- Driving anger, sensation seeking, impulsiveness, and boredom proneness in the prediction of unsafe drivingAccident Analysis & Prevention, 2005
- A comparative study of texture measures with classification based on featured distributionsPattern Recognition, 1996