Convolution-Based Neural Attention With Applications to Sentiment Classification
Authors:
Open Access
- 21 February 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 7, 27983-27992
Abstract
Neural attention mechanism has achieved many successes in various tasks in natural language processing. However, existing neural attention models based on a densely connected network are loosely related to the attention mechanism found in psychology and neuroscience. Motivated by the finding in neuroscience that human possesses the template-searching attention mechanism, we propose to use convolution operation to simulate attentions and give a mathematical explanation of our neural attention model. We then introduce a new network architecture, which combines a recurrent neural network with our convolution-based attention model and further stacks an attention-based neural model to build a hierarchical sentiment classification model. The experimental results show that our proposed models can capture salient parts of the text to improve the performance of sentiment classification at both the sentence level and the document level.Keywords
Funding Information
- National Natural Science Foundation of China (U1636103, 61632011, 61876053)
- Key Technologies Research and Development Program of Shenzhen (JSGG20170817140856618)
- Shenzhen Foundational Research Funding (20170307150024907)
- Innovate UK (103652)
This publication has 34 references indexed in Scilit:
- An adaptive hybrid pattern for noise-robust texture analysisPattern Recognition, 2015
- Sentic patterns: Dependency-based rules for concept-level sentiment analysisKnowledge-Based Systems, 2014
- Local Metric Learning for Exemplar-Based Object DetectionIEEE Transactions on Circuits and Systems for Video Technology, 2014
- Sentiment Analysis and Opinion MiningSynthesis Lectures on Human Language Technologies, 2012
- Absence of real evidence against competition during syntactic ambiguity resolutionJournal of Memory and Language, 2006
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Long Short-Term MemoryNeural Computation, 1997
- Simple statistical gradient-following algorithms for connectionist reinforcement learningMachine Learning, 1992
- Inference during reading.Psychological Review, 1992
- Visual search and stimulus similarity.Psychological Review, 1989