FecTek: Enhancing Term Weight in Lexicon-Based Retrieval with Feature Context and Term-level Knowledge
Preprint
Abstract
Lexicon-based retrieval has gained siginificant popularity in text retrieval due to its efficient and robust performance. To further enhance performance of lexicon-based retrieval, researchers have been diligently incorporating state-of-the-art methodologies like Neural retrieval and text-level contrastive learning approaches. Nonetheless, despite the promising outcomes, current lexicon-based retrieval methods have received limited attention in exploring the potential benefits of feature context representations and term-level knowledge guidance. In this paper, we introduce an innovative method by introducing FEature Context and TErm-level Knowledge modules(FecTek). To effectively enrich the feature context representations of term weight, the Feature Context Module (FCM) is introduced, which leverages the power of BERT's representation to determine dynamic weights for each element in the embedding. Additionally, we develop a term-level knowledge guidance module (TKGM) for effectively utilizing term-level knowledge to intelligently guide the modeling process of term weight. Evaluation of the proposed method on MS Marco benchmark demonstrates its superiority over the previous state-of-the-art approaches.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction2022
- Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval2022
- RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking2021
- Condenser: a Pre-training Architecture for Dense Retrieval2021
- PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval2021
- COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List2021
- RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering2021
- Biased TextRank: Unsupervised Graph-Based Content Extraction2020
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019