Evaluating whole-page relevance

Authors:

Abstract
Whole page relevance defines how well the surface-level repre- sentation of all elements on a search result page and the corre- sponding holistic attributes of the presentation respond to users' information needs. We introduce a method for evaluating the whole-page relevance of Web search engine results pages. Our key contribution is that the method allows us to investigate as- pects of component relevance that are difficult or impossible to judge in isolation. Such aspects include component-level infor- mation redundancy and cross-component coherence. The method we describe complements traditional document relevance meas- urement, affords comparative relevance assessment across multi- ple search engines, and facilitates the study of important factors such as brand presentation effects and component-level quality.