Towards Understanding and Answering Comparative Questions
Authors:
Open Access
- 11 February 2022
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 66-74
Abstract
In this paper, we analyze comparative questions and answers. At least 3%~of the questions submitted to search engines are comparative; ranging from simple facts like "Did Messi or Ronaldo score more goals in 2021?'' to life-changing and probably highly subjective questions like "Is it better to move abroad or stay?''. Ideally, answers to subjective comparative questions would reflect diverse opinions so that the asker can come to a well-informed decision. To better understand the information needs behind comparative questions, we develop approaches to extract the mentioned comparison objects and aspects. As a first step to answer comparative questions, we develop an approach that detects the stances of potential result nuggets (i.e., text passages containing the comparison objects). Our approaches are trained and evaluated on a set of 31,000~English questions from existing datasets that we label as comparative or not. In the 3,500~comparative questions, we label the comparison objects, aspects, and predicates. For 950~questions, we collect answers from online forums and label the stance towards the comparison objects. In the experiments, our approaches recall~71% of the comparative questions with a perfect precision of~1.0, recall~92% of subjective comparative questions with a precision of~0.98, and identify the comparison objects and aspects with an F1 of~0.93 and~0.80, respectively. The stance detector fine-tuned on pairs of objects and answers achieves an accuracy of~0.63.Keywords
Funding Information
- Ministry of Science and Higher Education of the Russian Federation (075-02-2021-1387)
- Deutsche Forschungsgemeinschaft (HA 5851/2-1 and HA 5851/2-2)
This publication has 12 references indexed in Scilit:
- Transformers: State-of-the-Art Natural Language Processing2020
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension2020
- Stanza: A Python Natural Language Processing Toolkit for Many Human Languages2020
- Natural Questions: A Benchmark for Question Answering ResearchTransactions of the Association for Computational Linguistics, 2019
- HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering2018
- Cross-topic Argument Mining from Heterogeneous Sources2018
- An analysis of comparison questions in the context of auditingDiscourse Processes, 1990
- INTERPRETING ENGLISH COMPARATIVESJournal of Semantics, 1986
- THE COMPARATIVE COMPAREDJournal of Semantics, 1984
- Features of similarity.Psychological Review, 1977