Methods for ranking information retrieval systems without relevance judgments
Authors:
Open Access
- 9 March 2003
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 811-816
Abstract
In this paper we present some new methods of ranking information retrieval systems without relevance judgement. The common ground of these methods is using a measure we called reference count. An extensive experimentation was conducted to evaluate the effectiveness of the proposed methods using various different standards Information Retrieval evaluation measures for the ranking, like average precision, R-precision, and precision and different document levels. We also compared the effectiveness of the proposed methods with the method proposed by Soboroff et al. The experimental results showed that the proposed methods are effective, and in many cases are more effective than Soboroff at al.'s method.Keywords
This publication has 9 references indexed in Scilit:
- Data fusion with estimated weights2002
- Ranking retrieval systems without relevance judgments2001
- Fusion Via a Linear Combination of ScoresInformation Retrieval Journal, 1999
- Automatic discovery of language models for text databases1999
- How reliable are the results of large-scale information retrieval experiments?1998
- Efficient construction of large test collections1998
- Analyses of multiple evidence combination1997
- The text retrieval conferences (TRECS)1996
- Combining the evidence of multiple query representations for information retrievalInformation Processing & Management, 1995