MLTCP: A Distributed Technique to Approximate Centralized Flow Scheduling For Machine Learning
Open Access
- 18 November 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 167-176
Abstract
This paper argues that congestion control protocols in machine learning datacenters sit at a sweet spot between centralized and distributed flow scheduling solutions. We present MLTCP, a technique to augment today's congestion control algorithms to approximate an interleaved centralized flow schedule. At the heart of MLTCP lies a straight-forward principle based on a key conceptual insight: by scaling the congestion window size (or sending rate) based on the number of bytes sent at each iteration, MLTCP flows eventually converge into a schedule that reduces network contention. We demonstrate that MLTCP uses a gradient descent trend with a step taken at every training (or fine-tuning) iteration towards reducing network congestion among competing jobs.Keywords
Funding Information
- DARPA FastNICs (4202290027)
- NSF (National Science Foundation) SHF (2107244)
- NSF (National Science Foundation) CNS (2211382)
- NSF (National Science Foundation) PPoSS (2217099)
- NSF (National Science Foundation) CAREER (2144766)
- Sloan School of Management, Massachusetts Institute of Technology Fellowship (FG-2022-18504)
This publication has 40 references indexed in Scilit:
- PIAS: Practical Information-Agnostic Flow Scheduling for Commodity Data CentersIEEE/ACM Transactions on Networking, 2017
- DX: Latency-Based Congestion Control for DatacentersIEEE/ACM Transactions on Networking, 2016
- CUBICACM SIGOPS Operating Systems Review, 2008
- TCP-Illinois: A loss- and delay-based congestion control algorithm for high-speed networksPerformance Evaluation, 2008
- Binary increase congestion control (BIC) for fast long-distance networks2005
- Processor Sharing Flows in the Internet2005
- Scheduling of Offset Free SystemsReal-Time Systems, 2003
- TCP Westwood: congestion window control using bandwidth estimation2001
- TCP Vegas: end to end congestion avoidance on a global InternetIEEE Journal on Selected Areas in Communications, 1995
- A Mathematical Model for Periodic Scheduling ProblemsSIAM Journal on Discrete Mathematics, 1989