Representation Learning for Question Classification via Topic Sparse Autoencoder and Entity Embedding
- 1 December 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 126-133
Abstract
Deep learning models have achieved great successes these days. There are intensive studies of word representation learning for question classification. As questions are typically short texts, existing techniques are often not effective for extracting discriminative representations of questions just from a limited number of words. This motivates us to exploit additional information beyond words in order to improve the representation learning of questions. On one hand, topic modeling often captures meaningful semantic structures from the question corpus. Such global topical information should be helpful for question representations. On the other hand, entities extracted from question themselves provide more auxiliary information for short texts from a local viewpoint. Together with words, topics and entities, question representations can be substantially improved.In this paper, we propose a unified neural network framework by integrating Topic modeling, Word embedding and Entity Embedding (TWEE) for question representation learning. Concretely, we introduce a novel topic sparse autoencoder to incorporate discriminative topics into the representation learning of questions. In addition, both words and entity related information are embedded into the network to help learn a more comprehensive question representation. Empirical experiments show that the proposed TWEE framework outperforms the state-of-the-art methods on different datasets.Keywords
This publication has 16 references indexed in Scilit:
- A comparative study of neural network models for lexical intent classification2015
- Document Modeling with Gated Recurrent Neural Network for Sentiment Classification2015
- Deep Unordered Composition Rivals Syntactic Methods for Text Classification2015
- Dependency-based Convolutional Neural Networks for Sentence Embedding2015
- Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks2015
- NCR2014
- A Convolutional Neural Network for Modelling Sentences2014
- Glove: Global Vectors for Word Representation2014
- Convolutional Neural Networks for Sentence Classification2014
- From symbolic to sub-symbolic information in question classificationArtificial Intelligence Review, 2010