Server selection methods in hybrid portal search
Authors:
Open Access
- 15 August 2005
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 75-82
Abstract
The TREC.GOV collection makes a valuable web testbed for distributed information retrieval methods because it is naturally partitioned and includes 725 web-oriented queries with judged answers. It can usefully model aspects of government and large corporate portals. Analysis of the.gov data shows that a purely distributed approach would not be feasible for providing search on a.gov portal because of the large number (17,000+) of web sites and the high proportion that do not provide a search interface. An alternative hybrid approach, combining both distributed and centralized techniques, is proposed and server selection methods are evaluated within this framework using web-oriented evaluation methodology. A number of well-known algorithms are compared against representatives (highest anchor ranked page (HARP) and anchor weighted sum (AWSUM)) of a family of new selection methods which use link anchortext extracted from an auxiliary crawl to provide descriptions of sites which are not themselves crawled. Of the previously published methods, ReDDE substantially outperformed three variants of CORI and also outperformed a method based on Kullback-Leibler Divergence (extended) except on topic distillation. HARP and AWSUM performed best overall but were outperformed on the topic distillation task by extended KL Divergence.Keywords
This publication has 19 references indexed in Scilit:
- The perfect search engine is not enough2004
- Combining CORI and the Decision-Theoretic Approach for Advanced Resource Selection2004
- Comparing the performance of collection selection algorithmsACM Transactions on Information Systems, 2003
- A taxonomy of web searchACM SIGIR Forum, 2002
- Approaches to collection selection and results merging for distributed information retrieval2001
- A case study in web search using TREC algorithms2001
- Server selection on the World Wide Web2000
- Analysis of a very large web search engine query logACM SIGIR Forum, 1999
- GlOSSACM Transactions on Database Systems, 1999
- Searching distributed collections with inference networks1995