Task-Driven Comparison of Topic Models
Authors:
- 13 August 2015
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 22 (1) , 320-329
Abstract
Topic modeling, a method of statistically extracting thematic content from a large collection of texts, is used for a wide variety of tasks within text analysis. Though there are a growing number of tools and techniques for exploring single models, comparisons between models are generally reduced to a small set of numerical metrics. These metrics may or may not reflect a model's performance on the analyst's intended task, and can therefore be insufficient to diagnose what causes differences between models. In this paper, we explore task-centric topic model comparison, considering how we can both provide detail for a more nuanced understanding of differences and address the wealth of tasks for which topic models are used. We derive comparison tasks from single-model uses of topic models, which predominantly fall into the categories of understanding topics, understanding similarity, and understanding change. Finally, we provide several visualization techniques that facilitate these tasks, including buddy plots, which combine color and position encodings to allow analysts to readily view changes in document similarity.Keywords
Funding Information
- NSF (IIS-1162037)
This publication has 23 references indexed in Scilit:
- ModelTracker2015
- High-Reproducibility and High-Accuracy Method for Automated Topic ClassificationPhysical Review X, 2015
- Opening the Black Box: Strategies for Increased User Involvement in Existing Algorithm ImplementationsIEEE Transactions on Visualization and Computer Graphics, 2014
- Visual Analysis of Topic Competition on Social MediaIEEE Transactions on Visualization and Computer Graphics, 2013
- UTOPIAN: User-Driven Topic Modeling Based on Interactive Nonnegative Matrix FactorizationIEEE Transactions on Visualization and Computer Graphics, 2013
- Coral: an integrated suite of visualizations for comparing clusteringsBMC Bioinformatics, 2012
- The Hundredth Psalm to the Tune of "Green Sleeves": Digital Approaches to Shakespeare's Language of GenreShakespeare Quarterly, 2010
- Topic Significance Ranking of LDA Generative Models2009
- ColorBrewer in Print: A Catalog of Color Schemes for MapsCartography and Geographic Information Science, 2003
- Indexing by latent semantic analysisJournal of the American Society for Information Science, 1990