Learning Query-aware Embedding Index for Improving E-commerce Dense Retrieval
Open Access
- 18 July 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 3265-3269
Abstract
The embedding index has become an essential part of the dense retrieval (DR) system, which enables a fast search for billion of items in online E-commerce applications. To accelerate the retrieval process in industrial scenarios, most of the previous studies only utilize item embeddings. However, the product quantization process without query embeddings will lead to inconsistency between queries and items. A straightforward solution is to put query embedding into the product quantization process. But we found that the distance of the positive query and item embedding pairs is too large, which means the query and item embeddings learned by the two-tower are not fully aligned. This problem would lead to performance decay when directly putting query embeddings into the product quantization. In this paper, we propose a novel query-aware embedding Index framework, which aligns the query and item embedding space to reduce the distance between positive pairs, thereby mixing the query and item embeddings to learn better cluster centers for product quantization. Specifically, we first propose s symmetric loss to train a better two-tower to achieve space alignment. Subsequently, we propose a mixed quantization strategy to put the query embeddings into the product quantization process for bridging the gap between queries and compressed item embeddings. Extensive experiments show that our framework significantly outperforms previous models on a real-world dataset, which demonstrates the superiority and effectiveness of the framework.Keywords
This publication has 17 references indexed in Scilit:
- Optimizing Dense Retrieval Model Training with Hard Negatives2021
- RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering2021
- Embedding-based Retrieval in Facebook Search2020
- ColBERT2020
- Towards Personalized and Semantic Retrieval: An End-to-End Solution for E-commerce Search via Embedding Learning2020
- Dense Passage Retrieval for Open-Domain Question Answering2020
- Billion-Scale Similarity Search with GPUsIEEE Transactions on Big Data, 2019
- Learning deep structured semantic models for web search using clickthrough data2013
- Locality-sensitive hashing scheme based on p-stable distributions2004
- Some Simple Effective Approximations to the 2-Poisson Model for Probabilistic Weighted Retrieval1994