Design and Evaluation of an RDMA-aware Data Shuffling Operator for Parallel Database Systems
Authors:
Open Access
- 12 December 2019
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Database Systems
- Vol. 44 (4) , 1-45
Abstract
The commoditization of high-performance networking has sparked research interest in the RDMA capability of this hardware. One-sided RDMA primitives, in particular, have generated substantial excitement due to the ability to directly access remote memory from within an application without involving the TCP/IP stack or the remote CPU. This article considers how to leverage RDMA to improve the analytical performance of parallel database systems. To shuffle data efficiently using RDMA, one needs to consider a complex design space that includes (1) the number of open connections, (2) the contention for the shared network interface, (3) the RDMA transport function, and (4) how much memory should be reserved to exchange data between nodes during query processing. We contribute eight designs that capture salient tradeoffs in this design space as well as an adaptive algorithm to dynamically manage RDMA-registered memory. We comprehensively evaluate how transport-layer decisions impact the query performance of a database system for different generations of InfiniBand. We find that a shuffling operator that uses the RDMA Send/Receive transport function over the Unreliable Datagram transport service can transmit data up to 4× faster than an RDMA-capable MPI implementation in a 16-node cluster. The response time of TPC-H queries improves by as much as 2×.Keywords
Funding Information
- National Science Foundation (SHF-1816577,III-1422977,III-1464381,CNS-1513120)
- Google Research Faculty Award
This publication has 28 references indexed in Scilit:
- Fast in-memory transaction processing using RDMA and HTM2015
- G ra M2015
- On the Design and Scalability of Distributed Shared-Data Databases2015
- Rack-Scale In-Memory Join Processing using RDMA2015
- Locality-sensitive operators for parallel main-memory database clusters2014
- Parallel join executions in RAMCloud2014
- Peak performance2013
- Efficiently compiling efficient query plans for modern hardwareProceedings of the VLDB Endowment, 2011
- High Performance RDMA-Based MPI Implementation over InfiniBandInternational Journal of Parallel Programming, 2004
- Volcano-an extensible and parallel query evaluation systemIEEE Transactions on Knowledge and Data Engineering, 1994