Mining Quality Phrases from Massive Text Corpora
Open Access
- 27 May 2015
- conference paper
- Published by Association for Computing Machinery (ACM)
- Vol. 2015, 1729-1744
Abstract
Text data are ubiquitous and play an essential role in big data applications. However, text data are mostly unstructured. Transforming unstructured text into structured units (e.g., semantically meaningful phrases) will substantially reduce semantic ambiguity and enhance the power and efficiency at manipulating such data using database technology. Thus mining quality phrases is a critical research problem in the field of databases. In this paper, we propose a new framework that extracts quality phrases from text corpora integrated with phrasal segmentation. The framework requires only limited training but the quality of phrases so generated is close to human judgment. Moreover, the method is scalable: both computation time and required space grow linearly as corpus size increases. Our experiments on large text corpora demonstrate the quality and efficiency of the new method.Keywords
Funding Information
- U.S. Army Research Lab (W911NF-09-2-0053)
- National Science Foundation (IIS-1017362 IIS-1320617 and IIS-1354329)
- DTRA (HDTRA1-10-1-0120)
- NIH Big Data to Knowledge (BD2K) (1U54GM114838)
This publication has 22 references indexed in Scilit:
- Scalable topical phrase mining from text corporaProceedings of the VLDB Endowment, 2014
- Automatic Construction and Ranking of Topical Keyphrases on Collections of Short Documents2014
- Active LearningSynthesis Lectures on Artificial Intelligence and Machine Learning, 2012
- Interesting-phrase mining for ad-hoc text analyticsProceedings of the VLDB Endowment, 2010
- Towards the web of conceptsProceedings of the VLDB Endowment, 2010
- Meme-tracking and the dynamics of the news cycle2009
- Topic Cube: Topic Modeling for OLAP on Multidimensional Text Databases2009
- Multidimensional content eXplorationProceedings of the VLDB Endowment, 2008
- Non-projective dependency parsing using spanning tree algorithms2005
- KEA1999