Multidimensional Extra Evidence Mining for Image Sentiment Analysis
Authors:
Open Access
- 1 June 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 103619-103634
Abstract
Image sentiment analysis is a hot research topic in the field of computer vision. However, two key issues need to be addressed. First, high-quality training samples are scarce. There are numerous ambiguous images in the original datasets owing to diverse subjective cognitions from different annotators. Second, the cross-modal sentimental semantics among heterogeneous image features has not been fully explored. To alleviate these problems, we propose a novel model called multidimensional extra evidence mining (ME2M) for image sentiment analysis, it involves sample-refinement and cross-modal sentimental semantics mining. A new soft voting-based sample-refinement strategy is designed to address the former problem, whereas the state-of-the-art discriminant correlation analysis (DCA) model is used to completely mine the cross-modal sentimental semantics among diverse image features. Image sentiment analysis is conducted based on the cross-modal sentimental semantics and a general classifier. The experimental results verify that the ME2M model is effective and robust and that it outperforms the most competitive baselines on two well-known datasets. Furthermore, it is versatile owing to its flexible structure.Keywords
Funding Information
- National Natural Science Foundation of China (61762038, 61861016)
- Jiangxi Provincial Department of Science and Technology (20171BAB202023, 20192ACB21004)
- Key Research and Development Plan of the Jiangxi Provincial Science and Technology Department (20171BBG70093, 20192BBE50071)
- Humanity and Social Fund of Ministry of Education of China (17YJAZH117, 20YJAZH142)
- Humanity and Social Science Foundation of Jiangxi University (TQ19101)
- Science and Technology Projects of the Jiangxi Provincial Department of Education (GJJ180320, GJJ190323)
- Foundation of Guizhou Provincial Key Laboratory of Public Big Data (2018BDKFJJ011)
This publication has 38 references indexed in Scilit:
- Semi-Supervised Cross-Media Feature Learning With Unified Patch Graph RegularizationIEEE Transactions on Circuits and Systems for Video Technology, 2015
- Cross-modal Retrieval with Correspondence Autoencoder2014
- Large-scale visual sentiment ontology and detectors using adjective noun pairs2013
- Analyzing and predicting sentiment of images on the social web2010
- A Fast Learning Algorithm for Deep Belief NetsNeural Computation, 2006
- Histograms of Oriented Gradients for Human Detection2005
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Robust Real-Time Face DetectionInternational Journal of Computer Vision, 2004
- Multiresolution gray-scale and rotation invariant texture classification with local binary patternsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2002
- Introduction to Hotelling (1936) Relations Between Two Sets of Variates1992