Abstract
This paper argues that congestion control protocols in machine learning datacenters sit at a sweet spot between centralized and distributed flow scheduling solutions. We present MLTCP, a technique to augment today's congestion control algorithms to approximate an interleaved centralized flow schedule. At the heart of MLTCP lies a straight-forward principle based on a key conceptual insight: by scaling the congestion window size (or sending rate) based on the number of bytes sent at each iteration, MLTCP flows eventually converge into a schedule that reduces network contention. We demonstrate that MLTCP uses a gradient descent trend with a step taken at every training (or fine-tuning) iteration towards reducing network congestion among competing jobs.
Funding Information
  • DARPA FastNICs (4202290027)
  • NSF (National Science Foundation) SHF (2107244)
  • NSF (National Science Foundation) CNS (2211382)
  • NSF (National Science Foundation) PPoSS (2217099)
  • NSF (National Science Foundation) CAREER (2144766)
  • Sloan School of Management, Massachusetts Institute of Technology Fellowship (FG-2022-18504)

This publication has 40 references indexed in Scilit: