Independent factor topic models
Authors:
Open Access
- 14 June 2009
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 833-840
Abstract
Topic models such as Latent Dirichlet Allo- cation (LDA) and Correlated Topic Model (CTM) have recently emerged as powerful statistical tools for text document modeling. In this paper, we improve upon CTM and propose Independent Factor Topic Models (IFTM) which use linear latent variable mod- els to uncover the hidden sources of correla- tion between topics. There are 2 main contri- butions of this work. First, by using a sparse source prior model, we can directly visual- ize sparse patterns of topic correlations. Sec- ondly, the conditional independence assump- tion implied in the use of latent source vari- ables allows the objective function to factor- ize, leading to a fast Newton-Raphson based variational inference algorithm. Experimen- tal results on synthetic and real data show that IFTM runs on average 3-5 times faster than CTM, while giving competitive perfor- mance as measured by perplexity and log- likelihood of held-out data.Keywords
This publication has 4 references indexed in Scilit:
- Finding scientific topicsProceedings of the National Academy of Sciences, 2004
- A Variational Method for Learning Sparse and Overcomplete RepresentationsNeural Computation, 2001
- An Introduction to Latent Variable Models1984
- Some Contributions to Maximum Likelihood Factor AnalysisPsychometrika, 1967