X-RDMA: Effective RDMA Middleware in Large-scale Production Environments
Authors:
- 1 September 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-12
Abstract
X-RDMA is a communication middleware deployed and heavily used in Alibaba’s large-scale cluster hosting cloud storage and database systems. Unlike recent research projects which purely focus on squeezing out the raw hardware performance, it puts emphasis on robustness, scalability and maintainability of large-scale production clusters. X-RDMA integrates necessary features, not available in current RDMA ecosystem, to release the developers from complex and imperfect details. X-RDMA simplifies the programming model, extends RDMA protocols for application awareness, and proposes mechanisms for resource management with thousands of connections per machine. It also reduces the work for administration and performance tuning with built-in tracing, tuning and monitoring tools.X-RDMA has been deployed in several large-scale clusters with over 4000 servers in Alibaba cloud since 2016. It can save at least 70% development and maintenance time over RDMA, effectively improve performance and reduce network jitter especially when production servers are under pressure. It also helped locate over 30 issues in different layers of productions with over 5000 connections for each server on average.Keywords
This publication has 38 references indexed in Scilit:
- G ra M2015
- Inside the Social Network's (Datacenter) Network2015
- Pingmesh2015
- Congestion Control for Large-Scale RDMA Deployments2015
- UCX: An Open Source Framework for HPC Network APIs and Beyond2015
- Optimizing latency and CPU load in packet processing systems2015
- Designing a High Performance OpenSHMEM Implementation Using Universal Common Communication Substrate as a Communication MiddlewareLecture Notes in Computer Science, 2014
- High Performance RDMA Protocols in HPC2006
- High performance RDMA-based MPI implementation over InfiniBand2003
- NetSTAT: a network-based intrusion detection approach2002