A Graph Convolutional Topic Model for Short and Noisy Text Streams
Preprint
Abstract
Learning hidden topics from data streams has become absolutely necessary but posed challenging problems such as concept drift as well as short and noisy data. Using prior knowledge to enrich a topic model is one of potential solutions to cope with these challenges. Prior knowledge that is derived from human knowledge (e.g. Wordnet) or a pre-trained model (e.g. Word2vec) is very valuable and useful to help topic models work better. However, in a streaming environment where data arrives continually and infinitely, existing studies are limited to exploiting these resources effectively. Especially, a knowledge graph, that contains meaningful word relations, is ignored. In this paper, to aim at exploiting a knowledge graph effectively, we propose a novel graph convolutional topic model (GCTM) which integrates graph convolutional networks (GCN) into a topic model and a learning method which learns the networks and the topic model simultaneously for data streams. In each minibatch, our method not only can exploit an external knowledge graph but also can balance the external and old knowledge to perform well on new data. We conduct extensive experiments to evaluate our method with both a human knowledge graph (Wordnet) and a graph built from pre-trained word embeddings (Word2vec). The experimental results show that our method achieves significantly better performances than state-of-the-art baselines in terms of probabilistic predictive measure and topic coherence. In particular, our method can work well when dealing with short texts as well as concept drift. The implementation of GCTM is available at \url{https://github.com/bachtranxuan/GCTM.git}.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Dynamic Transformation of Prior Knowledge Into Bayesian Models for Data StreamsIEEE Transactions on Knowledge and Data Engineering, 2021
- Bag of biterms modeling for short textsKnowledge and Information Systems, 2020
- Eliminating overfitting of probabilistic topic models on short and noisy text: The role of dropoutInternational Journal of Approximate Reasoning, 2019
- Improving interpretations of topic modeling in microblogsJournal of the Association for Information Science and Technology, 2017
- Enhancing Topic Modeling for Short Texts with Auxiliary Word EmbeddingsACM Transactions on Information Systems, 2017
- A general framework to expand short text for topic modelingInformation Sciences, 2017
- A topic model for co-occurring normal documents and short textsWorld Wide Web, 2017
- An effective and interpretable method for document classificationKnowledge and Information Systems, 2016
- BTM: Topic Modeling over Short TextsIEEE Transactions on Knowledge and Data Engineering, 2014
- The Latent Process Decomposition of cDNA Microarray Data SetsIEEE/ACM Transactions on Computational Biology and Bioinformatics, 2005