Rethinking Intra-host Congestion Control in RDMA Networks
Open Access
- 3 August 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 31-37
Abstract
RDMA has been widely deployed in production datacenters. The conventional wisdom believes that the intra-host network delivers stable and high performance. However, intra-host resources witness a relative stagnation in technology trends compared to the evolving RDMA NIC (RNIC). Thus, the RNIC traffic may not get sufficient intra-host resources when it contends with intra-host traffic. A line of recent works from large-scale production datacenter operators demonstrates the emergence of intra-host congestion and associated performance collapse, which forces us to rethink the practice of intra-host congestion control. However, the ability to efficiently control RDMA intra-host networks is far less mature than inter-host networks, which brings challenges in congestion monitoring, intra-host resource allocation and RNIC traffic adjustment. In this paper, we propose RDMA intra-Host Congestion Control (RHCC), which combines sub-RTT granularity intra-host traffic congestion avoidance and proactive RNIC traffic adjustment. We implement RHCC on commodity servers and RNICs and conduct experiments to evaluate the performance. The results show that RHCC can increase/decrease the network throughput/latency by up to 2 × and 1.4 ×, respectively.Keywords
This publication has 10 references indexed in Scilit:
- Host Congestion Control2023
- Towards a Manageable Intra-Host Network2023
- Compute Express Link (CXL): Enabling Heterogeneous Data-Centric Computing With Heterogeneous Memory HierarchyIEEE Micro, 2022
- Understanding host interconnect congestion2022
- Understanding I/O Direct Cache Access Performance for End Host Networking2022
- EFLOPS: Algorithm and System Co-Design for a High Performance Distributed Training Platform2020
- Understanding PCIe performance for end host networking2018
- Congestion Control for Large-Scale RDMA DeploymentsACM SIGCOMM Computer Communication Review, 2015
- Accelerating Spark with RDMA for Big Data Processing: Early Experiences2014
- High-Performance Design of Hadoop RPC with RDMA over InfiniBand2013