PetaKV: Building Efficient Key-Value Store for File System Metadata on Persistent Memory
- 27 December 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 34 (3) , 843-855
Abstract
Previous works proposed building file systems and organizing the metadata with KV stores because KV stores handle entries of various sizes efficiently and have excellent scalability. The emergence of the byte-addressable persistent memory (PM) enables metadata service to be faster than before by tailoring the KV store for the PM. However, existing PM-based KV stores cannot handle the workloads of file systems’ metadata well because simply depending on hash tables or trees cannot simultaneously provide fast file accessing and efficient directory traversing. In this paper, we exploit the insight of the metadata operations and propose the PetaKV, a KV store tailored for the metadata management of file systems on PM. PetaKV leverages dual hash indexing to achieve fast file put and get operations. Moreover, it cooperates with PM-tailored peta logs to collocate KV entries for each directory, thus supporting efficient directory scans. Our evaluation indicates PetaKV outperforms state-of-art tree-based KV stores on put, get and scan $2.5\times$ , $3.2\times$ , and $2.8\times$ on average, respectively. Moreover, the file system built with PetaKV achieves $1.2\times$ to $6.4\times$ speedup compared to those built with tree-based KV stores on the metadata operations.
Keywords
Funding Information
- Creative Research Group Project of NSFC (61821003)
- National Natural Science Foundation of China (62072196, 62102155)
- Special Project for Research and Development in Key areas of Guangdong Province (2021B0101400003)
This publication has 16 references indexed in Scilit:
- Enabling low tail latency on multicore key-value storesProceedings of the VLDB Endowment, 2020
- LB+TreesProceedings of the VLDB Endowment, 2020
- FlatStore: An Efficient Log-Structured Key-Value Storage Engine for Persistent Memory2020
- An Efficient and Flexible Metadata Management Layer for Local File Systems2019
- A Flattened Metadata Service for Distributed File SystemsIEEE Transactions on Parallel and Distributed Systems, 2018
- FASTER2018
- The Snowflake Elastic Data Warehouse2016
- Asynchronized ConcurrencyACM SIGARCH Computer Architecture News, 2015
- IndexFS: Scaling File System Metadata Performance with Stateless Caching and Bulk Insertion2014
- Benchmarking cloud serving systems with YCSB2010