Result diversification based on query‐specific cluster ranking
- 18 January 2011
- journal article
- research article
- Published by Wiley in Journal of the American Society for Information Science and Technology
- Vol. 62 (3) , 550-571
Abstract
Result diversification is a retrieval strategy for dealing with ambiguous or multi‐faceted queries by providing documents that cover as many facets of the query as possible. We propose a result diversification framework based on query‐specific clustering and cluster ranking, in which diversification is restricted to documents belonging to clusters that potentially contain a high percentage of relevant documents. Empirical results show that the proposed framework improves the performance of several existing diversification methods. The framework also gives rise to a simple yet effective cluster‐based approach to result diversification that selects documents from different clusters to be included in a ranked list in a round robin fashion. We describe a set of experiments aimed at thoroughly analyzing the behavior of the two main components of the proposed diversification framework, ranking and selecting clusters for diversification. Both components have a crucial impact on the overall performance of our framework, but ranking clusters plays a more important role than selecting clusters. We also examine properties that clusters should have in order for our diversification framework to be effective. Most relevant documents should be contained in a small number of high‐quality clusters, while there should be no dominantly large clusters. Also, documents from these high‐quality clusters should have a diverse content. These properties are strongly correlated with the overall performance of the proposed diversification framework.Keywords
This publication has 33 references indexed in Scilit:
- An effective coherence measure to determine topical consistency in user-generated contentInternational Journal on Document Analysis and Recognition (IJDAR), 2009
- Mobile information retrieval with search results clustering: Prototypes and evaluationsJournal of the American Society for Information Science and Technology, 2009
- A risk minimization framework for information retrievalInformation Processing & Management, 2006
- The effectiveness of query-specific hierarchic clustering in information retrievalInformation Processing & Management, 2002
- 10.1162/jmlr.2003.3.4-5.993Applied Physics Letters, 2000
- Recent trends in hierarchic document clustering: A critical reviewInformation Processing & Management, 1988
- Beyond topicalityInformation Processing & Management, 1982
- A model of cluster searching based on classificationInformation Systems, 1980
- The use of hierarchic clustering in information retrievalInformation Storage and Retrieval, 1971
- A searching procedure for information retrievalInformation Storage and Retrieval, 1964