Classification of Patient Emotions in Nonverbal Communication Based on Machine Learning
- 26 September 2023
- journal article
- research article
- Published by Pleiades Publishing Ltd in Pattern Recognition and Image Analysis
- Vol. 33 (3) , 360-367
Abstract
This article is devoted to solving the problem of classifying a patient’s emotions in nonverbal communication based on machine learning. The article reflects the current state of the problem in the field of classification of human emotions in nonverbal communication and gives the formulation of the problem and the results of experimental studies for finding the most high-quality model that allows for the classification of emotions. The resulting model has an architecture that includes two recurrent subnets with an attention mechanism, the outputs of which are combined and fed to a fully connected classification layer. As a result, the obtained model on the validation dataset has accuracy of 0.77. The model was developed in Python using the TensorFlow and Keras frameworks. To extract the image of the face, the BlazeFace model from the MediaPipe framework was used.Keywords
This publication has 13 references indexed in Scilit:
- A Proposal for Multimodal Emotion Recognition Using Aural Transformers and Action Units on RAVDESS DatasetApplied Sciences, 2021
- Multimodal Emotion Recognition on RAVDESS Dataset Using Transfer LearningSensors, 2021
- Multi-Modal Fusion Emotion Recognition Method of Speech Expression Based on Deep LearningFrontiers in Neurorobotics, 2021
- The Experienced Level of Stress and Anxiety in Patients of a Multidisciplinary Medical CenterRussian Sklifosovsky Journal "Emergency Medical Care", 2020
- РАСПОЗНАВАНИЕ ЭМОЦИЙ ПО ИЗОБРАЖЕНИЮ ЛИЦА НА ОСНОВЕ ГЛУБОКИХ НЕЙРОННЫХ СЕТЕЙ2020
- The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American EnglishPLOS ONE, 2018
- Deep Residual Learning for Image Recognition2016
- Non-verbal communication in GermanyModern Communication Studies, 2013
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Long Short-Term MemoryNeural Computation, 1997