Xin Liu
Research Subjects
Xin Liu has a total of 101 co-authors. Here are the top co-authors by number of shared publications:
- Haibin Lin
ByteDance Inc., Seattle, USA
7 shared publications - Yanghua Peng
ByteDance Inc., Seattle, USA
4 shared publications - Ziheng Jiang
ByteDance Seed, Seattle, USA
3 shared publications - Borui Wan
The University of Hong Kong, Hong Kong, Hong Kong
2 shared publications - Chao Jin
School of Computer Science, Peking University, Beijing, China
2 shared publications - Chengquan Jiang
ByteDance Seed, Bellevue, WA, USA
2 shared publications
Latest Publications
- Open Access
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 366-382
MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
We present MegaScale-MoE, a production system tailored for the efficient training of large-scale mixture-of-experts (MoE) models. MoE emerges as a promising architecture to scale large language models (LLMs) to unprecedented sizes, thereby enhancing model performance. However, existing MoE training systems experience a degradation in training efficiency, exacerbated by the escalating scale of MoE models and the continuous evolution of hardware. Recognizing the pivotal role of efficient communication in enhancing MoE training, MegaScale-MoE customizes communication-efficient parallelism strategies for attention and FFNs in each MoE layer and adopts a holistic approach to overlap communication with computation at both inter- and intra-operator levels. Additionally, MegaScale-MoE applies communication compression with adjusted communication patterns to lower precision, further improving training efficiency. When training a 352B MoE model on 1,440 NVIDIA Hopper GPUs, MegaScale-MoE achieves a training throughput of 1.41M tokens/s, improving the efficiency by 1.88X compared to Megatron-LM. We share our operational experience in accelerating MoE training and hope that by offering our insights in system design, this work will motivate future research in MoE systems.
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 366-382
Open AccessMegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
We present MegaScale-MoE, a production system tailored for the efficient training of large-scale mixture-of-experts (MoE) models. MoE emerges as a promising architecture to scale large language models (LLMs) to unprecedented sizes, thereby enhancing model performance. However, existing MoE training systems experience a degradation in training efficiency, exacerbated by the escalating scale of MoE models and the continuous evolution of hardware. Recognizing the pivotal role of efficient communication in enhancing MoE training, MegaScale-MoE customizes communication-efficient parallelism strategies for attention and FFNs in each MoE layer and adopts a holistic approach to overlap communication with computation at both inter- and intra-operator levels. Additionally, MegaScale-MoE applies communication compression with adjusted communication patterns to lower precision, further improving training efficiency. When training a 352B MoE model on 1,440 NVIDIA Hopper GPUs, MegaScale-MoE achieves a training throughput of 1.41M tokens/s, improving the efficiency by 1.88X compared to Megatron-LM. We share our operational experience in accelerating MoE training and hope that by offering our insights in system design, this work will motivate future research in MoE systems.
- Open Access
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 400-422
Laminar: A Scalable Asynchronous RL Post-Training Framework
Reinforcement learning (RL) post-training for Large Language Models (LLMs) is now scaling to large clusters and running for extended durations to enhance model reasoning performance. However, the scalability of existing RL frameworks is limited, as extreme long-tail skewness in RL trajectory generation causes severe GPU underutilization. Current asynchronous RL systems attempt to mitigate this, but they rely on global weight synchronization between the actor and all rollouts, which creates a rigid model update schedule. This global synchronization is ill-suited for the highly skewed and evolving distribution of trajectory generation latency in RL training, crippling training efficiency. Our key insight is that efficient scaling requires breaking this lockstep through trajectory-level asynchrony, which generates and consumes each trajectory independently. We propose Laminar, a scalable and robust RL post-training system built on a fully decoupled architecture. First, we replace global updates with a tier of relay workers acting as a distributed parameter service. This enables asynchronous and fine-grained weight synchronization, allowing rollouts to pull the latest weight anytime without stalling the actor's training loop. Second, a dynamic repack mechanism consolidates long-tail trajectories onto a few dedicated rollouts, maximizing generation throughput. The fully decoupled design also isolates failures, ensuring robustness for long-running jobs. Our evaluation on a 1024–GPU cluster shows that Laminar achieves up to 5.48X training throughput speedup over state-of-the-art systems, while reducing model convergence time.
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 400-422
Open AccessLaminar: A Scalable Asynchronous RL Post-Training Framework
Reinforcement learning (RL) post-training for Large Language Models (LLMs) is now scaling to large clusters and running for extended durations to enhance model reasoning performance. However, the scalability of existing RL frameworks is limited, as extreme long-tail skewness in RL trajectory generation causes severe GPU underutilization. Current asynchronous RL systems attempt to mitigate this, but they rely on global weight synchronization between the actor and all rollouts, which creates a rigid model update schedule. This global synchronization is ill-suited for the highly skewed and evolving distribution of trajectory generation latency in RL training, crippling training efficiency. Our key insight is that efficient scaling requires breaking this lockstep through trajectory-level asynchrony, which generates and consumes each trajectory independently. We propose Laminar, a scalable and robust RL post-training system built on a fully decoupled architecture. First, we replace global updates with a tier of relay workers acting as a distributed parameter service. This enables asynchronous and fine-grained weight synchronization, allowing rollouts to pull the latest weight anytime without stalling the actor's training loop. Second, a dynamic repack mechanism consolidates long-tail trajectories onto a few dedicated rollouts, maximizing generation throughput. The fully decoupled design also isolates failures, ensuring robustness for long-running jobs. Our evaluation on a 1024–GPU cluster shows that Laminar achieves up to 5.48X training throughput speedup over state-of-the-art systems, while reducing model convergence time.
- Open Access
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 675-692
MegaScale-Omni: A Hyper-Scale, Workload-Resilient System for MultiModal LLM Training in Production
As the foundational component of versatile AI applications, training an multimodal large language model (MLLM) relies on multimodal datasets with dynamic modality mixture proportions and sample length distributions. However, existing MLLM systems remain inefficient under dynamic workloads, due to statically coupled decisions of resource allocation and model parallelization between encoders and the LLM backbone. This paper presents MegaScale-Omni, an industrial-grade MLLM training system tailored for dynamic workload adaption and hyper-scale deployment. MegaScale-Omni is built upon the training scheme of encoder-LLM multiplexing with three key innovations: (1) Decoupled parallelism strategies with long-short sequence parallelism for encoders to process variable-length samples, and full-fledged 5D parallelism for the LLM backbone, both organized under a communication-efficient parallelization layout. (2) Unified encoder-LLM representations for flexible, extensible colocation, and a new paradigm of encoder-LLM joint pipeline with workload resilience. (3) Workload balancing techniques via decentralized grouped reordering in data loaders and adaptive resharding from encoder to LLM ranks. MegaScale-Omni is deployed as the foundation of our in-house large-scale MLLM training tasks with thousands of GPUs. Our experimental results demonstrate 1.27×–7.57× throughput improvement under production-grade dynamic workloads, as compared to four state-of-the-art systems.
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 675-692
Open AccessMegaScale-Omni: A Hyper-Scale, Workload-Resilient System for MultiModal LLM Training in Production
As the foundational component of versatile AI applications, training an multimodal large language model (MLLM) relies on multimodal datasets with dynamic modality mixture proportions and sample length distributions. However, existing MLLM systems remain inefficient under dynamic workloads, due to statically coupled decisions of resource allocation and model parallelization between encoders and the LLM backbone. This paper presents MegaScale-Omni, an industrial-grade MLLM training system tailored for dynamic workload adaption and hyper-scale deployment. MegaScale-Omni is built upon the training scheme of encoder-LLM multiplexing with three key innovations: (1) Decoupled parallelism strategies with long-short sequence parallelism for encoders to process variable-length samples, and full-fledged 5D parallelism for the LLM backbone, both organized under a communication-efficient parallelization layout. (2) Unified encoder-LLM representations for flexible, extensible colocation, and a new paradigm of encoder-LLM joint pipeline with workload resilience. (3) Workload balancing techniques via decentralized grouped reordering in data loaders and adaptive resharding from encoder to LLM ranks. MegaScale-Omni is deployed as the foundation of our in-house large-scale MLLM training tasks with thousands of GPUs. Our experimental results demonstrate 1.27×–7.57× throughput improvement under production-grade dynamic workloads, as compared to four state-of-the-art systems.
- Open Access
- 22 March 2026
- Published by Association for Computing Machinery (ACM)
- p. 2197-2211
SwiftSpec: Disaggregated Speculative Decoding and Fused Kernels for Low-Latency LLM Inference
- 22 March 2026
- Published by Association for Computing Machinery (ACM)
- p. 2197-2211
Open AccessSwiftSpec: Disaggregated Speculative Decoding and Fused Kernels for Low-Latency LLM Inference
- Open Access
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1619-1630
LiquidGEMM: Hardware-Efficient W4A8 GEMM Kernel for High-Performance LLM Serving
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1619-1630
Open AccessLiquidGEMM: Hardware-Efficient W4A8 GEMM Kernel for High-Performance LLM Serving
- Open Access
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 186-203
Robust LLM Training Infrastructure at ByteDance
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 186-203
Open AccessRobust LLM Training Infrastructure at ByteDance
- Open Access
- 27 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 592-608
MegaScale-Infer: Efficient Mixture-of-Experts Model Serving with Disaggregated Expert Parallelism
- 27 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 592-608
Open AccessMegaScale-Infer: Efficient Mixture-of-Experts Model Serving with Disaggregated Expert Parallelism
- Open Access
- 27 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 963-978
ByteScale: Communication-Efficient Scaling of LLM Training with a 2048K Context Length on 16384 GPUs
- 27 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 963-978
Open AccessByteScale: Communication-Efficient Scaling of LLM Training with a 2048K Context Length on 16384 GPUs