A Survey of Storage Systems in the RDMA Era
- 12 July 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 33 (12) , 4395-4409
Abstract
Remote Direct Memory Access (RDMA) based network devices are increasingly being deployed in modern data centers. RDMA brings significant performance improvements over traditional network devices such as Ethernet due to its unique features: protocol offloading and memory semantics . In particular, it can achieve microsecond level latency, which is about 2 $\sim$ 3 orders of magnitude improvement. With such improvement in hardware, the software stack, including device drivers and programming libraries, is becoming a new performance bottleneck. Developers need to use new programming libraries to take full advantage of the performance of the underlying hardware. Storage systems are very important in modern data centers. This article surveys the current efforts to use RDMA for optimizing storage systems. We first present five classes of RDMA-based storage systems, including key-value stores, file systems, distributed memory systems, database systems, and systems using smart NICs, to demonstrate different design choices. Then, we examine the core modules of storage systems from different perspectives: communication mode, concurrency control, fault tolerance, caching, and resource management. Finally, we provide some design guidelines for new RDMA-based storage systems, as well as a discussion of opportunities and challenges.
Keywords
Funding Information
- National Key Research & Development Program of China (2020YFC1522702)
- National Natural Science Foundation of China (62141216, 61877035)
- Tsinghua University
This publication has 80 references indexed in Scilit:
- HydraDB2015
- Fast in-memory transaction processing using RDMA and HTM2015
- The RAMCloud Storage SystemACM Transactions on Computer Systems, 2015
- Congestion Control for Large-Scale RDMA Deployments2015
- DARE2015
- RStore: A Direct-Access DRAM-based Data Store2015
- Performance analysis of NVMe SSDs and their implication on real world databases2015
- Mojim2015
- DaRPC2014
- High performance support of parallel virtual file system (PVFS2) over Quadrics2005