Beyond Boundaries: A Human-like Approach for Question Answering over Structured and Unstructured Information Sources
Open Access
- 11 June 2024
- journal article
- research article
- Published by MIT Press in Transactions of the Association for Computational Linguistics
- Vol. 12, 786-802
Abstract
Answering factual questions from heterogenous sources, such as graphs and text, is a key capacity of intelligent systems. Current approaches either (i) perform question answering over text and structured sources as separate pipelines followed by a merge step or (ii) provide an early integration, giving up the strengths of particular information sources. To solve this problem, we present “HumanIQ”, a method that teaches language models to dynamically combine retrieved information by imitating how humans use retrieval tools. Our approach couples a generic method for gathering human demonstrations of tool use with adaptive few-shot learning for tool augmented models. We show that HumanIQ confers significant benefits, including i) reducing the error rate of our strongest baseline (GPT-4) by over 50% across 3 benchmarks, (ii) improving human preference over responses from vanilla GPT-4 (45.3% wins, 46.7% ties, 8.0% loss), and (iii) outperforming numerous task-specific baselines.Keywords
This publication has 28 references indexed in Scilit:
- Lost in the Middle: How Language Models Use Long ContextsTransactions of the Association for Computational Linguistics, 2024
- Explainable Conversational Question Answering over Heterogeneous Sources via Iterative Graph Neural Networks2023
- Conversational Question Answering on Heterogeneous Sources2022
- QALD-9-plus: A Multilingual Dataset for Question Answering over DBpedia and Wikidata Translated by Native Speakers2022
- Natural Questions: A Benchmark for Question Answering ResearchTransactions of the Association for Computational Linguistics, 2019
- Neural architecture for question answering using a knowledge graph and web corpusInformation Retrieval Journal, 2019
- Hybrid Question Answering Using Heuristic Methods and Linked Data Schema2018
- When a Knowledge Base Is Not Enough2016
- HAWK – Hybrid Question Answering Using Linked Data2015
- WikidataCommunications of the ACM, 2014