PASE: PostgreSQL Ultra-High-Dimensional Approximate Nearest Neighbor Search Extension
Authors:
Open Access
- 31 May 2020
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2241-2253
Abstract
Similarity search has been widely used in various fields, particularly in the Alibaba ecosystem. The open-source solutions to a similarity search of vectors can only support a query with a single vector, whereas real-life scenarios generally require a processing of compound queries. Moreover, existing open-source implementations only provide runtime libraries, which have difficulty meeting the requirements of industrial applications. To address these issues, we designed a novel scheme for extending the index-type of PostgreSQL (PG), which enables a similar vector search and achieves a high-performance level and strong reliability of PG. Two representative types of nearest neighbor search (NNS) algorithms are presented herein. These algorithms achieve a high performance, and afford advantages such as the support of composite queries and seamless integration of existing business data. The other NNS algorithms can be easily implemented under the proposed framework. Experiments were conducted on large datasets to illustrate the efficiency of the proposed retrieval mechanism.Keywords
This publication has 4 references indexed in Scilit:
- Fast approximate nearest neighbor search with the navigating spreading-out graphProceedings of the VLDB Endowment, 2019
- The Inverted Multi-IndexIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Product Quantization for Nearest Neighbor SearchIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010
- Locality-sensitive hashing scheme based on p-stable distributions2004