CFE2: Counterfactual Editing for Search Result Explanation
Open Access
- 2 August 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 145-155
Abstract
Search Result Explanation (SeRE) aims to improve search sessions' effectiveness and efficiency by helping users interpret documents' relevance. Existing works mostly focus on factual explanation, i.e. to find/generate supporting evidence about documents' relevance to search queries. However, research in cognitive sciences has shown that human explanations are contrastive i.e. people explain an observed event using some counterfactual events; such explanations reduce cognitive load and provide actionable insights. Though already proven effective in machine learning and NLP communities, there lacks a strict formulation on how counterfactual explanations should be defined and structured, in the context of web search. In this paper, we first discuss the possible formulation of counterfactual explanations in the IR context. Next, we formulate a suite of desiderata for counterfactual explanation in SeRE task and corresponding automatic metrics. With this desiderata, we propose a method named CounterFactual Editing for Search Research Explanation (CFE2). CFE2 provides pairwise counterfactual explanations for document pairs within a search engine result page. Our experiments on five public search datasets demonstrate that CFE2can significantly outperform baselines in both automatic metrics and human evaluations.Keywords
All Related Versions
This publication has 52 references indexed in Scilit:
- Pretrained Transformers for Text Ranking2022
- Search, sense making and learning: closing gapsInformation and Learning Sciences, 2019
- WWW'18 Open Challenge2018
- Efficient Index-Based Snippet GenerationACM Transactions on Information Systems, 2014
- Exploring information seeking processes in collaborative search tasksProceedings of the American Society for Information Science and Technology, 2010
- Fast generation of result snippets in web search2007
- Query Recommendation Using Query Logs in Search Engines2004
- Advantages of query biased summaries in information retrieval1998
- Contrastive ExplanationRoyal Institute of Philosophy Supplement, 1990
- Conversational processes and causal explanation.Psychological Bulletin, 1990