A Performance Study to Guide RDMA Programming Decisions
Authors:
- Patrick MacArthurPMPatrick MacArthur
- Robert D. RussellRRRobert D. Russell
- 1 June 2012
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 778-785
Abstract
This paper describes a performance study of Remote Direct Memory Access (RDMA) programming techniques. Its goal is to use these results as a guide for making “best practice” RDMA programming decisions. Infiniband RDMA is widely used in scientific high performance computing (HPC) clusters as a low-latency, high-bandwidth, reliable interconnect accessed via MPI. Recently it is gaining adherents outside scientific HPC as high-speed clusters appear in other application areas for which MPI is not suitable. RDMA enables user applications to move data directly between virtual memory on different nodes without operating system intervention, so there is a need to know how to incorporate RDMA access into high-level programs. But RDMA offers more options to a programmer than traditional sockets programming, and it is not always obvious what the performance tradeoffs of these options might be. This study is intended to provide some answers.Keywords
This publication has 10 references indexed in Scilit:
- High Performance Data Transfer in Grid Environment Using GridFTP over InfiniBand2010
- Designing Efficient FTP Mechanisms for High Performance Data-Transfer over InfiniBand2009
- Early Experiences with Write-Write Design of NFS over RDMA2009
- Design and evaluation of benchmarks for financial applications using Advanced Message Queuing Protocol (AMQP) over InfiniBand2008
- Performance Analysis and Evaluation of PCIe 2.0 and Quad-Data Rate InfiniBand2008
- Direct Data Placement over Reliable Transports2007
- Marker PDU Aligned Framing for TCP Specification2007
- A Remote Direct Memory Access Protocol Specification2007
- Reducing Connection Memory Requirements of MPI for InfiniBand Clusters: A Message Coalescing Approach2007
- PVFS over InfiniBand: design and performance evaluation2003