From paragraph to graph: Latent semantic analysis for information visualization
- 6 April 2004
- journal article
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 101 (suppl_1) , 5214-5219
Abstract
Most techniques for relating textual information rely on intellectually created links such as author-chosen keywords and titles, authority indexing terms, or bibliographic citations. Similarity of the semantic content of whole documents, rather than just titles, abstracts, or overlap of keywords, offers an attractive alternative. Latent semantic analysis provides an effective dimension reduction method for the purpose that reflects synonymy and the sense of arbitrary word combinations. However, latent semantic analysis correlations with human text-to-text similarity judgments are often empirically highest at ≈300 dimensions. Thus, two- or three-dimensional visualizations are severely limited in what they can show, and the first and/or second automatically discovered principal component, or any three such for that matter, rarely capture all of the relations that might be of interest. It is our conjecture that linguistic meaning is intrinsically and irreducibly very high dimensional. Thus, some method to explore a high dimensional similarity space is needed. But the 2.7 × 107projections and infinite rotations of, for example, a 300-dimensional pattern are impossible to examine. We suggest, however, that the use of a high dimensional dynamic viewer with an effective projection pursuit routine and user control, coupled with the exquisite abilities of the human visual system to extract information about objects and from moving patterns, can often succeed in discovering multiple revealing views that are missed by current computational algorithms. We show some examples of the use of latent semantic analysis to support such visualizations and offer views on future needs.Keywords
This publication has 9 references indexed in Scilit:
- Mixed-membership models of scientific publicationsProceedings of the National Academy of Sciences, 2004
- An unsupervised method for the extraction of propositional information from textProceedings of the National Academy of Sciences, 2004
- Finding scientific topicsProceedings of the National Academy of Sciences, 2004
- Visualizing knowledge domainsAnnual Review of Information Science and Technology, 2003
- XGobi: Interactive Dynamic Data Visualization in the X Window SystemJournal of Computational and Graphical Statistics, 1998
- An introduction to latent semantic analysisDiscourse Processes, 1998
- A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge.Psychological Review, 1997
- Indexing by latent semantic analysisJournal of the American Society for Information Science, 1990
- Formative design evaluation of superbookACM Transactions on Information Systems, 1989