A multimodal sentiment analysis system for recognizing person aggressiveness in pain based on textual and visual information
Open Access
- 2 March 2023
- journal article
- research article
- Published by Springer Nature in Journal of Ambient Intelligence and Humanized Computing
- Vol. 14 (4) , 4489-4501
Abstract
This article proposes a multimodal sentiment analysis system for recognizing a person’s aggressiveness in pain. The implementation has been divided into five components. The first three steps are related to a text-based sentiment analysis system to perform classification tasks such as predicting the classes into non-aggressive, covertly aggressive, and overtly aggressive classes. The remaining two components are related to an image-based sentiment analysis system. A deep learning-based approach has been employed to do feature learning and predict the three types of pain classes. An aggression dataset for the text-based system and the UNBC-McMaster database for an image-based system has been employed, respectively. Experimental results have been compared with the state-of-the-art methods, showing the superiority of the proposed approach. Finally, the scores due to text-based and image-based sentiment analysis systems are fused to obtain the performance for the proposed multimodal sentiment analysis system.Keywords
Funding Information
- Università degli Studi di Salerno
This publication has 49 references indexed in Scilit:
- A Lexicon-based Approach for Hate Speech DetectionInternational Journal of Multimedia and Ubiquitous Engineering, 2015
- Cyber Hate Speech on Twitter: An Application of Machine Classification and Statistical Modeling for Policy and Decision MakingPolicy & Internet, 2015
- Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Experts and Machines against Bullies: A Hybrid Approach to Detect Cyberbullies2014
- Improving Cyberbullying Detection with User Context2013
- Mixed emotions: Holistic and analytic perception of facial expressionsJournal of Vision, 2009
- What is a support vector machine?Nature Biotechnology, 2006
- Classifying racist texts using a support vector machine2004
- Multimodal human emotion/expression recognition2002
- Long Short-Term MemoryNeural Computation, 1997