Text Classification with Topic-based Word Embedding and Convolutional Neural Networks
Authors:
Open Access
- 2 October 2016
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 88-97
Abstract
Recently, distributed word embeddings trained by neural language models are commonly used for text classification with Convolutional Neural Networks (CNNs). In this paper, we propose a novel neural language model, Topic-based Skip-gram, to learn topic-based word embeddings for biomedical literature indexing with CNNs. Topic-based Skip-gram leverages textual content with topic models, e.g., Latent Dirichlet Allocation (LDA), to capture precise topic-based word relationship and then integrate it into distributed word embedding learning. We then describe two multimodal CNN architectures, which are able to employ different kinds of word embeddings at the same time for text classification. Through extensive experiments conducted on several real-world datasets, we demonstrate that combination of our Topic-based Skip-gram and multimodal CNN architectures outperforms state-of-the-art methods in biomedical literature indexing, clinical note annotation and general textual benchmark dataset classification.Keywords
This publication has 14 references indexed in Scilit:
- Convolutional neural networks for biomedical text classification2015
- MeSHLabeler: improving the accuracy of large-scale MeSH indexing by integrating diverse evidenceBioinformatics, 2015
- A Convolutional Neural Network for Modelling Sentences2014
- Glove: Global Vectors for Word Representation2014
- Convolutional Neural Networks for Sentence Classification2014
- Latent Dirichlet Allocation Models for Image ClassificationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- A Bayesian Hierarchical Model for Learning Natural Scene Categories2005
- Producing high-dimensional semantic spaces from lexical co-occurrenceBehavior Research Methods, Instruments, & Computers, 1996
- NewsWeeder: Learning to Filter Netnews1995
- Indexing by latent semantic analysisJournal of the American Society for Information Science, 1990