Learning Topic Models: Identifiability and Finite-Sample Analysis
Authors:
- 19 July 2022
- journal article
- research article
- Published by Taylor & Francis in Journal of the American Statistical Association
- Vol. 118 (544) , 2860-2875
Abstract
Topic models provide a useful text-mining tool for learning, extracting, and discovering latent structures in large text corpora. Although a plethora of methods have been proposed for topic modeling, lacking in the literature is a formal theoretical investigation of the statistical identifiability and accuracy of latent topic estimation. In this paper, we propose a maximum likelihood estimator (MLE) of latent topics based on a specific integrated likelihood that is naturally connected to the concept, in computational geometry, of volume minimization. Our theory introduces a new set of geometric conditions for topic model identifiability, conditions that are weaker than conventional separability conditions, which typically rely on the existence of pure topic documents or of anchor words. Weaker conditions allow a wider and thus potentially more fruitful investigation. We conduct finite-sample error analysis for the proposed estimator and discuss connections between our results and those of previous investigations. We conclude with empirical studies employing both simulated and real datasets.Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- Blind Separation of Quasi-Stationary Sources: Exploiting Convex Geometry in Covariance DomainIEEE Transactions on Signal Processing, 2015
- Using mixture models for collaborative filteringJournal of Computer and System Sciences, 2008
- Estimating the endpoint of a distribution in the presence of additive observation errorsStatistics & Probability Letters, 2004
- Finding scientific topicsProceedings of the National Academy of Sciences of the United States of America, 2004
- Latent Semantic Indexing: A Probabilistic AnalysisJournal of Computer and System Sciences, 2000
- The Stochastic EM Algorithm: Estimation and Asymptotic ResultsBernoulli, 2000
- 10.1162/jmlr.2003.3.4-5.993Applied Physics Letters, 2000
- Integrated likelihood methods for eliminating nuisance parametersStatistical Science, 1999
- On the complexity of four polyhedral set containment problemsMathematical Programming, 1985
- Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraumstrahlung)Mathematische Annalen, 1912