Parameterized generation of labeled datasets for text categorization based on a hierarchical directory
Open Access
- 25 July 2004
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 250-257
Abstract
Although text categorization is a burgeoning area of IR research, readily available test collections in this field are surprisingly scarce. We describe a methodology and system (named ACCIO) for automatically acquiring labeled datasets for text categorization from the World Wide Web, by capitalizing on the body of knowledge encoded in the structure of existing hierarchical directories such as the Open Directory. We define parameters of categories that make it possible to acquire numerous datasets with desired properties, which in turn allow better control over categorization experiments. In particular, we develop metrics that estimate the difficulty of a dataset by examining the host directory structure. These metrics are shown to be good predictors of categorization accuracy that can be achieved on a dataset, and serve as efficient heuristics for generating datasets subject to user's requirements. A large collection of automatically generated datasets are made available for other researchers to use.Keywords
This publication has 18 references indexed in Scilit:
- Text categorization with many redundant features2004
- Automatic Association of Web Directories with Word SensesComputational Linguistics, 2003
- Personalized pocket directories for mobile devices2002
- The structure of broad topics on the web2002
- Concept Hierarchy-Based Text Database CategorizationKnowledge and Information Systems, 2002
- Machine learning in automated text categorizationACM Computing Surveys, 2002
- Hierarchical classification of Web content2000
- NewsWeeder: Learning to Filter Netnews1995
- The DARPA TIPSTER projectACM SIGIR Forum, 1992
- Evaluating text categorization1991