Bayesian Sparse Topical Coding
- 1 June 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 31 (6) , 1080-1093
Abstract
Sparse topic models (STMs) are widely used for learning a semantically rich latent sparse representation of short texts in large scale, mainly by imposing sparse priors or appropriate regularizers on topic models. However, it is difficult for STMs to model a sparse structure and pattern of a corpora accurately, since their sparse priors always fail to achieve real sparseness, and regularizers bypass the prior information of the relevance between sparse coefficients. In this paper, we propose a novel Bayesian hierarchical topic model called Bayesian Sparse Topical Coding with Poisson Distribution (BSTC-P) on the basis of Sparse Topical Coding with Sparse Groups (STCSG). Different from traditional STMs, it focuses on imposing hierarchical sparse prior to leverage the prior information of the relevance between sparse coefficients. Furthermore, we propose a sparsity-enhanced BSTC, Bayesian Sparse Topical Coding with Normal Distribution (BSTC-N), via mathematic approximation. We adopt superior hierarchical sparse inducing prior, with the purpose of achieving the sparsest optimal solution. Experimental results on datasets of Newsgroups and Twitter show that the BSTC-N outperforms BSTC-P on finding clear latent semantic representations, and both BSTC-P and BSTC-N have better performance than existing works on document classification tasks.Keywords
Funding Information
- National Natural Science Foundation of China (61472291)
- National Natural Science Foundation of China (61272110)
- National Natural Science Foundation of China (41472288)
This publication has 43 references indexed in Scilit:
- A Sparse-Group LassoJournal of Computational and Graphical Statistics, 2013
- Lexical association measures and collocation extractionLanguage Resources and Evaluation, 2009
- Bayesian Inference for Nonnegative Matrix Factorisation ModelsComputational Intelligence and Neuroscience, 2009
- Nonnegative Matrix Factorization with Gaussian Process PriorsComputational Intelligence and Neuroscience, 2008
- Sparse Principal Component AnalysisJournal of Computational and Graphical Statistics, 2006
- Model Selection and Estimation in Regression with Grouped VariablesJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005
- Sparse Bayesian Learning for Basis SelectionIEEE Transactions on Signal Processing, 2004
- Adaptive sparseness for supervised learningIEEE Transactions on Pattern Analysis and Machine Intelligence, 2003
- The expectation-maximization algorithmIEEE Signal Processing Magazine, 1996
- Indexing by latent semantic analysisJournal of the American Society for Information Science, 1990