Graph Convolutional Topic Model for Data Streams
Preprint
Abstract
Learning hidden topics in data streams has been paid a great deal of attention by researchers with a lot of proposed methods, but exploiting prior knowledge in general and a knowledge graph in particular has not been taken into adequate consideration in these methods. Prior knowledge that is derived from human knowledge (e.g. Wordnet) or a pre-trained model (e.g.Word2vec) is very valuable and useful to help topic models work better, especially on short texts. However, previous work often ignores this resource, or it can only utilize prior knowledge of a vector form in a simple way. In this paper, we propose a novel graph convolutional topic model (GCTM) which integrates graph convolutional networks (GCN) into a topic model and a learning method which learns the networks and the topic model simultaneously for data streams. In each minibatch, our method not only can exploit an external knowledge graph but also can balance between the external and old knowledge to perform well on new data. We conduct extensive experiments to evaluate our method with both human graph knowledge(Wordnet) and a graph built from pre-trained word embeddings (Word2vec). The experimental results show that our method achieves significantly better performances than the state-of-the-art baselines in terms of probabilistic predictive measure and topic coherence. In particular, our method can work well when dealing with short texts as well as concept drift. The implementation of GCTM is available at https://github.com/bachtranxuan/GCTM.git.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Collaborative Topic Model for Poisson distributed ratingsInternational Journal of Approximate Reasoning, 2018
- Improving interpretations of topic modeling in microblogsJournal of the Association for Information Science and Technology, 2017
- Overcoming catastrophic forgetting in neural networksProceedings of the National Academy of Sciences of the United States of America, 2017
- Graph Convolutional Encoders for Syntax-aware Neural Machine Translation2017
- An effective and interpretable method for document classificationKnowledge and Information Systems, 2016
- The Latent Process Decomposition of cDNA Microarray Data SetsIEEE/ACM Transactions on Computational Biology and Bioinformatics, 2005