Efficient diversification of web search results
Open Access
- 1 April 2011
- journal article
- Published by Association for Computing Machinery (ACM) in Proceedings of the VLDB Endowment
- Vol. 4 (7) , 451-459
Abstract
In this paper we analyze the efficiency of various search results diversification methods. While efficacy of diversification approaches has been deeply investigated in the past, response time and scalability issues have been rarely addressed. A unified framework for studying performance and feasibility of result diversification solutions is thus proposed. First we define a new methodology for detecting when, and how, query results need to be diversified. To this purpose, we rely on the concept of "query refinement" to estimate the probability of a query to be ambiguous . Then, relying on this novel ambiguity detection method, we deploy and compare on a standard test set, three different diversification methods: IASelect, xQuAD, and OptSelect. While the first two are recent state-of-the-art proposals, the latter is an original algorithm introduced in this paper. We evaluate both the efficiency and the effectiveness of our approach against its competitors by using the standard TREC Web diversification track testbed. Results shown that OptSelect is able to run two orders of magnitude faster than the two other state-of-the-art approaches and to obtain comparable figures in diversification effectiveness.Keywords
All Related Versions
This publication has 27 references indexed in Scilit:
- Explicit Search Result Diversification through Sub-queries2010
- A survey of Web clustering enginesACM Computing Surveys, 2009
- Predictive user click models based on click-through history2007
- Graphs from Search Engine Queries2007
- Improving personalized web search using result diversification2006
- A risk minimization framework for information retrievalInformation Processing & Management, 2006
- Improving recommendation lists through topic diversification2005
- Cumulated gain-based evaluation of IR techniquesACM Transactions on Information Systems, 2002
- The use of MMR, diversity-based reranking for reordering documents and producing summaries1998
- An analysis of approximations for maximizing submodular set functions—IMathematical Programming, 1978