Questions Are All You Need to Train a Dense Passage Retriever
Top Cited Papers
Open Access
- 20 June 2023
- journal article
- research article
- Published by MIT Press in Transactions of the Association for Computational Linguistics
- Vol. 11, 600-616
Abstract
We introduce ART, a new corpus-level autoencoding approach for training dense retrieval models that does not require any labeled training data. Dense retrieval is a central challenge for open-domain tasks, such as Open QA, where state-of-the-art methods typically require large supervised datasets with custom hard-negative mining and denoising of positive examples. ART, in contrast, only requires access to unpaired inputs and outputs (e.g., questions and potential answer passages). It uses a new passage-retrieval autoencoding scheme, where (1) an input question is used to retrieve a set of evidence passages, and (2) the passages are then used to compute the probability of reconstructing the original question. Training for retrieval based on question reconstruction enables effective unsupervised learning of both passage and question encoders, which can be later incorporated into complete Open QA systems without any further finetuning. Extensive experiments demonstrate that ART obtains state-of-the-art results on multiple QA retrieval benchmarks with only generic initialization from a pre-trained language model, removing the need for labeled data and task-specific losses.1 Our code and model checkpoints are available at: https://github.com/DevSinghSachan/art.Keywords
All Related Versions
This publication has 35 references indexed in Scilit:
- TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse LanguagesTransactions of the Association for Computational Linguistics, 2020
- ColBERT2020
- Dense Passage Retrieval for Open-Domain Question Answering2020
- Natural Questions: A Benchmark for Question Answering ResearchTransactions of the Association for Computational Linguistics, 2019
- Learning Dense Representations for Entity Retrieval2019
- Latent Retrieval for Weakly Supervised Open Domain Question Answering2019
- TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension2017
- SQuAD: 100,000+ Questions for Machine Comprehension of Text2016
- The Probabilistic Relevance Framework: BM25 and BeyondFoundations and Trends® in Information Retrieval, 2009
- SIGNATURE VERIFICATION USING A “SIAMESE” TIME DELAY NEURAL NETWORK1994