A Sequential Algorithm for Training Text Classifiers
Preprint
Abstract
The ability to cheaply train text classifiers is critical to their use in information retrieval, content analysis, natural language processing, and other tasks involving data which is partly or fully textual. An algorithm for sequential sampling during machine learning of statistical classifiers was developed and tested on a newswire text categorization task. This method, which we call uncertainty sampling, reduced by as much as 500-fold the amount of training data that would have to be manually classified to achieve a given level of effectiveness.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Selecting concise training sets from clean dataIEEE Transactions on Neural Networks, 1993
- A method for disambiguating word senses in a large corpusComputers and the Humanities, 1992
- Query-based learning applied to partially trained multilayer perceptronsIEEE Transactions on Neural Networks, 1991
- Models for retrieval with probabilistic indexingInformation Processing & Management, 1989
- Information retrieval: A sequential learning processJournal of the American Society for Information Science, 1983
- USING PROBABILISTIC MODELS OF DOCUMENT RETRIEVAL WITHOUT RELEVANCE INFORMATIONJournal of Documentation, 1979
- The condensed nearest neighbor rule (Corresp.)IEEE Transactions on Information Theory, 1968
- Automatic Indexing: An Experimental InquiryJournal of the ACM, 1961