An experiment in automatic indexing using the HASSET thesaurus
- 1 September 2013
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 13-18
Abstract
In this paper we present the tools, techniques and evaluation results of an automatic indexing experiment we conducted on the UK Data Archive/UK Data Service data-related document collection, as part of the Jisc-funded SKOS-HASSET project. We examined the quality of an automatic indexer based on a controlled vocabulary called the Humanities and Social Science Electronic Thesaurus (HASSET). We used the Keyphrase Extraction Algorithm (KEA), a text mining and a machine learning tool. KEA builds a classifier model using training documents with known keywords which is then applied to help assign keywords to new documents. We performed extensive manual and automatic evaluation on the results using recall, precision and F1 scores. The quality of the KEA indexing was measured a) automatically by the degree of overlap between the automated indexing decisions and those originally made by the human indexer and b) manually by comparing KEA's output with the source text. This paper explains how and why we applied the chosen technical solutions, and how we intend to take forward any lessons learned from this work in the future.Keywords
This publication has 5 references indexed in Scilit:
- An extensive experimental comparison of methods for multi-label learningPattern Recognition, 2012
- Interactive thesaurus assessment for automatic document annotation2007
- Using CISMeF MeSH “Encapsulated” terminology and a categorization algorithm for health resourcesInternational Journal of Medical Informatics, 2004
- In vitro evaluation of a program for machine-aided indexingInformation Processing & Management, 2002
- KEA1999