Beyond relevance and recall: testing new user‐centred measures of database performance
- 25 August 2009
- journal article
- Published by Wiley in Health Information and Libraries Journal
- Vol. 26 (3) , 220-231
Abstract
Measures of the effectiveness of databases have traditionally focused on recall, precision, with some debate on how relevance can be assessed, and by whom. New measures of database performance are required when users are familiar with search engines, and expect full text availability. This research ascertained which of four bibliographic databases (BNI, CINAHL, MEDLINE and EMBASE) could be considered most useful to nursing and midwifery students searching for information for an undergraduate dissertation. Searches on title were performed for dissertation topics supplied by nursing students (n = 9), who made the relevance judgements. Measures of recall and precision were combined with additional factors to provide measures of effectiveness, while efficiency combined measures of novelty and originality and accessibility combined measures for availability and retrievability, based on obtainability. There were significant differences among the databases in precision, originality and availability, but other differences were not significant (Friedman test). Odds ratio tests indicated that BNI, followed by CINAHL were the most effective, CINAHL the most efficient, and BNI the most accessible. The methodology could help library services in purchase decisions as the measure for accessibility, and odds ratio testing helped to differentiate database performance.Keywords
This publication has 23 references indexed in Scilit:
- The number needed to retrieve: a practically useful measure of information retrieval?Health Information and Libraries Journal, 2006
- Judgment of information quality and cognitive authority in the WebJournal of the American Society for Information Science and Technology, 2001
- Developing information literacy: a key to evidence‐based nursingInternational Nursing Review, 2001
- Recent relevance research: implications for information professionalsOnline Information Review, 2000
- Changes in relevance criteria and problem stages in task performanceJournal of Documentation, 2000
- Users' criteria for relevance evaluation: A cross-situational comparisonInformation Processing & Management, 1998
- THE BENEFITS OF SEARCHING EMBASE VERSUS MEDLINE FOR PHARMACEUTICAL INFORMATIONOnline and CD-Rom Review, 1998
- Selecting a database for literature searches in nursing: MEDLINE or CINAHL?Journal of Advanced Nursing, 1996
- How Easy are Randomized Controlled Trials in Epilepsy to Find on Medline? The Sensitivity and Precision of Two Medline SearchesEpilepsia, 1996
- Comparison of CINAHL, EMBASE, and MEDLINE Databases for the Nurse ResearcherMedical Reference Services Quarterly, 1993