SuccinctKV: a CPU-efficient LSM-tree Based KV Store with Scan-based Compaction
Open Access
- 20 November 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Architecture and Code Optimization
- Vol. 21 (4) , 1-26
Abstract
The CPU overhead of the LSM-tree becomes increasingly significant when high-speed storage devices are utilized. In this article, we propose SuccinctKV , a key-value store based on LSM-tree that is optimized to improve CPU efficiency in mixed workload scenarios. To achieve this, SuccinctKV reduces the CPU overhead of compaction by writing scan-sorted data directly to the storage device. SuccinctKV also redesigns the merge-sort operation of the LSM-tree, enhancing CPU locality and reducing the unnecessary CPU overhead of cache accesses and I/O system calls. Additionally, SuccinctKV introduces a scheduler to resolve potential bursty I/O contention by autonomously initiating I/O requests at appropriate times and quickly relieving I/O pressure by terminating background I/O requests. We implement SuccinctKV on RocksDB and conduct extensive experiments to evaluate our proposed methods. The experimental results demonstrate that, compared with RocksDB, SuccinctKV achieves a maximum improvement of 2.6x in scan performance and reduces CPU overhead of compaction by up to 89% under mixed workloads.Keywords
Funding Information
- National Key Research and Development Program of China (2023YFB4502905)
- NSFC (92270202)
- Natural Science Foundation of Shanghai Municipality (23ZR1418300)
This publication has 21 references indexed in Scilit:
- PM-Blade: A Persistent Memory Augmented LSM-tree Storage for Database2023
- Real-Time LSM-Trees for HTAP Workloads2023
- Designing a Cost-Effective Cache Replacement Policy using Machine Learning2021
- LSM-based storage techniques: a surveyThe VLDB Journal, 2019
- X-Engine2019
- SuRF2018
- Bridging the Archipelago between Row-Stores and Column-Stores for Hybrid Workloads2016
- ARIACM Transactions on Architecture and Code Optimization, 2013
- Benchmarking cloud serving systems with YCSB2010
- BigtableACM Transactions on Computer Systems, 2008