Finding scientific topics
Top Cited Papers
- 6 April 2004
- journal article
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences of the United States of America
- Vol. 101 (suppl_1) , 5228-5235
Abstract
A first step in identifying the content of a document is determining which topics that document addresses. We describe a generative model for documents, introduced by Blei, Ng, and Jordan [Blei, D. M., Ng, A. Y. & Jordan, M. I. (2003)J. Machine Learn. Res.3, 993-1022], in which each document is generated by choosing a distribution over topics and then choosing each word in the document from a topic selected according to this distribution. We then present a Markov chain Monte Carlo algorithm for inference in this model. We use this algorithm to analyze abstracts from PNAS by using Bayesian model selection to establish the number of topics. We show that the extracted topics capture meaningful structure in the data, consistent with the class designations provided by the authors of the articles, and outline further applications of this analysis, including identifying “hot topics” by examining temporal dynamics and tagging abstracts to illustrate semantic content.Keywords
This publication has 7 references indexed in Scilit:
- Unsupervised Learning by Probabilistic Latent Semantic AnalysisMachine Learning, 2001
- Monte Carlo Methods in Statistical Physics1999
- Markov Chain Monte Carlo in Practice1995
- Bayes FactorsJournal of the American Statistical Association, 1995
- Fundamental theorem of natural selection under gene-culture transmission.Proceedings of the National Academy of Sciences, 1991
- Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of ImagesIEEE Transactions on Pattern Analysis and Machine Intelligence, 1984
- Maximum Likelihood from Incomplete Data Via the EM AlgorithmJournal of the Royal Statistical Society Series B: Statistical Methodology, 1977