Probabilistic Topic and Syntax Modeling with Part-of-Speech LDA
Preprint
Abstract
This article presents a probabilistic generative model for text based on semantic topics and syntactic classes called Part-of-Speech LDA (POSLDA). POSLDA simultaneously uncovers short-range syntactic patterns (syntax) and long-range semantic patterns (topics) that exist in document collections. This results in word distributions that are specific to both topics (sports, education, ...) and parts-of-speech (nouns, verbs, ...). For example, multinomial distributions over words are uncovered that can be understood as "nouns about weather" or "verbs about law". We describe the model and an approximate inference algorithm and then demonstrate the quality of the learned topics both qualitatively and quantitatively. Then, we discuss an NLP application where the output of POSLDA can lead to strong improvements in quality: unsupervised part-of-speech tagging. We describe algorithms for this task that make use of POSLDA-learned distributions that result in improved performance beyond the state of the art.Keywords
All Related Versions
This publication has 4 references indexed in Scilit:
- A Two-Dimensional Topic-Aspect Model for Discovering Multi-Faceted TopicsProceedings of the AAAI Conference on Artificial Intelligence, 2010
- Unsupervised Learning by Probabilistic Latent Semantic AnalysisMachine Learning, 2001
- Overview of the third text Retrieval conference (TREC-3)1995
- Tagging text with a probabilistic model1991