Relevance-guided Supervision for OpenQA with ColBERT
Open Access
- 8 September 2021
- journal article
- research article
- Published by MIT Press in Transactions of the Association for Computational Linguistics
- Vol. 9, 929-944
Abstract
Systems for Open-Domain Question Answering (OpenQA) generally depend on a retriever for finding candidate passages in a large corpus and a reader for extracting answers from those passages. In much recent work, the retriever is a learned component that uses coarse-grained vector representations of questions and passages. We argue that this modeling choice is insufficiently expressive for dealing with the complexity of natural language questions. To address this, we define ColBERT-QA, which adapts the scalable neural retrieval model ColBERT to OpenQA. ColBERT creates fine-grained interactions between questions and passages. We propose an efficient weak supervision strategy that iteratively uses ColBERT to create its own training data. This greatly improves OpenQA retrieval on Natural Questions, SQuAD, and TriviaQA, and the resulting system attains state-of-the-art extractive OpenQA performance on all three datasets.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- Neural Ranking Models with Weak Supervision2017
- Reading Wikipedia to Answer Open-Domain Questions2017
- TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension2017
- My Computer Is an Honor Student — But How Intelligent Is It? Standardized Tests as a Measure of AIAI Magazine, 2016
- SQuAD: 100,000+ Questions for Machine Comprehension of Text2016
- WikiQA: A Challenge Dataset for Open-Domain Question Answering2015
- A Neural Network for Factoid Question Answering over Paragraphs2014
- Learning to Automatically Solve Algebra Word Problems2014
- Building Watson: An Overview of the DeepQA ProjectAI Magazine, 2010
- Building a question answering test collection2000