Donglin Li
Donglin Li has a total of 7 co-authors. Here are the top co-authors by number of shared publications:
Chunyi Cao
2 shared publications- Hanbo Ma
Department of Electrical and Electronic Engineering, The University of Manchester, Manchester, U.K.
2 shared publications Junan Wang
2 shared publications- Zhengtao Ding
Department of Electrical and Electronic Engineering, The University of Manchester, Manchester, U.K.
2 shared publications - Zhongguo Li
School of Automation, Southeast University, Nanjing, China
2 shared publications Jun Yang
1 shared publication
Latest Publications
- 3 September 2026
- Published by Elsevier in Neurocomputing
- Vol. 706
Load balancing for large language model inference serving: A control and learning oriented survey
- 3 September 2026
- Published by Elsevier in Neurocomputing
- Vol. 706
Load balancing for large language model inference serving: A control and learning oriented survey
- Open Access
- 18 June 2026
- Published by Elsevier in SSRN Electronic Journal
Load Balancing for Large Language Model Inference Serving: A Control and Learning Oriented Survey
Large language models (LLMs) are widely deployed as interactive neural inference services, where performance depends on how compute, memory, communication bandwidth, and persistent state are allocated under non-stationary demand. This survey studies load balancing for LLM inference serving as an intelligent scheduling and resource-allocation problem for large-scale learning systems. The analysis focuses on imbalance induced by prefill–decode phase asymmetry, key–value (KV) cache residency, heavy-tailed request variability, heterogeneous hardware, and service-level objectives (SLOs). A structured review protocol codes representative studies by imbalance source, observable signals, control knobs, assumptions, limitations, and evaluation evidence. We introduce a state– action–constraint abstraction, compare six strategy families, and synthesize evaluation practice around tail latency, time to first token (TTFT), token cadence, goodput, robustness, and energy per SLO-compliant token. Open directions are discussed for long-context and multimodal serving, decentralized deployment, green inference, and autonomous self-adaptive serving.
- 18 June 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessLoad Balancing for Large Language Model Inference Serving: A Control and Learning Oriented Survey
Large language models (LLMs) are widely deployed as interactive neural inference services, where performance depends on how compute, memory, communication bandwidth, and persistent state are allocated under non-stationary demand. This survey studies load balancing for LLM inference serving as an intelligent scheduling and resource-allocation problem for large-scale learning systems. The analysis focuses on imbalance induced by prefill–decode phase asymmetry, key–value (KV) cache residency, heavy-tailed request variability, heterogeneous hardware, and service-level objectives (SLOs). A structured review protocol codes representative studies by imbalance source, observable signals, control knobs, assumptions, limitations, and evaluation evidence. We introduce a state– action–constraint abstraction, compare six strategy families, and synthesize evaluation practice around tail latency, time to first token (TTFT), token cadence, goodput, robustness, and energy per SLO-compliant token. Open directions are discussed for long-context and multimodal serving, decentralized deployment, green inference, and autonomous self-adaptive serving.