Independent factor topic models

Authors:

Abstract
Topic models such as Latent Dirichlet Allo- cation (LDA) and Correlated Topic Model (CTM) have recently emerged as powerful statistical tools for text document modeling. In this paper, we improve upon CTM and propose Independent Factor Topic Models (IFTM) which use linear latent variable mod- els to uncover the hidden sources of correla- tion between topics. There are 2 main contri- butions of this work. First, by using a sparse source prior model, we can directly visual- ize sparse patterns of topic correlations. Sec- ondly, the conditional independence assump- tion implied in the use of latent source vari- ables allows the objective function to factor- ize, leading to a fast Newton-Raphson based variational inference algorithm. Experimen- tal results on synthetic and real data show that IFTM runs on average 3-5 times faster than CTM, while giving competitive perfor- mance as measured by perplexity and log- likelihood of held-out data.

This publication has 4 references indexed in Scilit: