Document Score Distribution Models for Query Performance Inference and Prediction
Author:
Open Access
- 1 January 2014
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
- Vol. 32 (1) , 1-28
Abstract
Modelling the distribution of document scores returned from an information retrieval (IR) system in response to a query is of both theoretical and practical importance. One of the goals of modelling document scores in this manner is the inference of document relevance. There has been renewed interest of late in modelling document scores using parameterised distributions. Consequently, a number of hypotheses have been proposed to constrain the mixture distribution from which document scores could be drawn. In this article, we show how a standard performance measure (i.e., average precision) can be inferred from a document score distribution using labelled data. We use the accuracy of the inference of average precision as a measure for determining the usefulness of a particular model of document scores. We provide a comprehensive study which shows that certain mixtures of distributions are able to infer average precision more accurately than others. Furthermore, we analyse a number of mixture distributions with regard to the recall-fallout convexity hypothesis and show that the convexity hypothesis is practically useful. Consequently, based on one of the best-performing score-distribution models, we develop some techniques for query-performance prediction (QPP) by automatically estimating the parameters of the document score-distribution model when relevance information is unknown. We present experimental results that outline the benefits of this approach to query-performance prediction.Keywords
This publication has 55 references indexed in Scilit:
- Variational bayes for modeling score distributionsInformation Retrieval Journal, 2010
- On the relationship between Spearman's rho and Kendall's tau for pairs of continuous random variablesJournal of Statistical Planning and Inference, 2007
- Query performance predictionInformation Systems, 2006
- Precision prediction based on ranked list coherenceInformation Retrieval Journal, 2006
- A study of smoothing methods for language models applied to information retrievalACM Transactions on Information Systems, 2004
- Probabilistic models of information retrieval based on measuring the divergence from randomnessACM Transactions on Information Systems, 2002
- Term-weighting approaches in automatic text retrievalInformation Processing & Management, 1988
- The incomplete Γ function as a continuous poisson distributionComputers & Mathematics with Applications, 1986
- When the most “pertinent” document should not be retrieved—An analysis of the Swets modelInformation Processing & Management, 1977
- Information Retrieval SystemsScience, 1963