D 2 Comp: Efficient Offload of LSM-tree Compaction with Data Processing Units on Disaggregated Storage
Open Access
- 14 September 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Architecture and Code Optimization
- Vol. 21 (3) , 1-22
Abstract
LSM-based key-value stores suffer from sub-optimal performance due to their slow and heavy background compactions. The compaction brings severe CPU and network overhead on high-speed disaggregated storage. This article further reveals that data-intensive compression in compaction consumes a significant portion of CPU power. Moreover, the multi-threaded compactions cause substantial CPU contention and network traffic during high-load periods. Based on the above observations, we propose fine-grained dynamical compaction offloading by leveraging the modern Data Processing Unit (DPU) to alleviate the CPU and network overhead. To achieve this, we first customized a file system to enable efficient data access for DPU. We then leverage the Arm cores on the DPU to meet the burst CPU and network requirements to reduce resource contention and data movement. We further employ dedicated hardware-based accelerators on the DPU to speed up the compression in compactions. We integrate our DPU-offloaded compaction with RocksDB and evaluate it with NVIDIA’s latest Bluefield-2 DPU on a real system. The evaluation shows that the DPU is an effective solution to solve the CPU bottleneck and reduce data traffic of compaction. The results show that compaction performance is accelerated by 2.86 to 4.03 times, system write and read throughput is improved by up to 3.2 times and 1.4 times respectively, and host CPU contention and network traffic are effectively reduced compared to the fine-tuned CPU-only baseline.Keywords
This publication has 25 references indexed in Scilit:
- TriangleKV: Reducing Write Stalls and Write Amplification in LSM-Tree Based KV Stores With Triangle Container in NVMIEEE Transactions on Parallel and Distributed Systems, 2022
- Xenic2021
- Supporting Elastic Compaction of LSM-tree with a FaaS Cluster2021
- Nova-LSM: A Distributed, Component-based LSM-tree Key-value Store2021
- Hailstorm2020
- File systems unfit as distributed storage backends2019
- KVell2019
- Offloading distributed applications onto smartNICs using iPipe2019
- SPDK: A Development Kit to Build High Performance Storage Applications2017
- The log-structured merge-tree (LSM-tree)Acta Informatica, 1996