RHCC: Revisiting Intra-Host Congestion Control in RDMA Networks
- 7 January 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Networking
- Vol. 33 (3) , 1189-1202
Abstract
RDMA has been widely deployed in production datacenters. The conventional wisdom believes that the intra-host network delivers stable and high performance. However, intra-host resources witness a relative stagnation in technology trends compared to the evolving RDMA NIC (RNIC). Thus, the RNIC traffic may not get sufficient intra-host resources when it contends with CPU-to-memory traffic. A line of recent works from large-scale production datacenter operators demonstrates the emergence of intra-host congestion and associated performance collapse, which forces us to revisit the practice of intra-host congestion control. However, the ability to efficiently control RDMA intra-host networks is far less mature than inter-host networks, which brings challenges in congestion monitoring, intra-host resource allocation and RNIC traffic adjustment. In this paper, we propose RDMA intra-Host Congestion Control (RHCC), which combines CPU-to-memory traffic congestion avoidance with sub-RTT granularity and proactive RNIC traffic adjustment. RHCC ensures fast congestion avoidance and can work with different inter-host congestion control methods. We implement RHCC on commodity servers and RNICs and conduct experiments to evaluate the performance. The results show that RHCC can increase/decrease the network throughput/latency by up to 2 $\times$ and 1.4 $\times$ , respectively.Keywords
Funding Information
- National Key Research and Development Program of China (2022YFB2901700)
- National Natural Science Foundation of China (62132022)
- Natural Science Foundation of Shandong Province (ZR2023LZH011)
- Program for Youth Innovative Research Team of Beijing University of Posts and Telecommunications (2024YQTD02)
This publication has 17 references indexed in Scilit:
- Compute Express Link (CXL): Enabling Heterogeneous Data-Centric Computing With Heterogeneous Memory HierarchyIEEE Micro, 2022
- Don’t Forget the I/O When Allocating Your LLC2021
- A review of PID control, tuning methods and applicationsInternational Journal of Dynamics and Control, 2020
- EFLOPS: Algorithm and System Co-Design for a High Performance Distributed Training Platform2020
- HPCC2019
- Understanding PCIe performance for end host networking2018
- TIMELYACM SIGCOMM Computer Communication Review, 2015
- Congestion Control for Large-Scale RDMA DeploymentsACM SIGCOMM Computer Communication Review, 2015
- Accelerating Spark with RDMA for Big Data Processing: Early Experiences2014
- High-Performance Design of Hadoop RPC with RDMA over InfiniBand2013