LST-Sim: An Efficient Simulation Platform for Large-Scale Model Training
Open Access
- 11 August 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1828-1832
Abstract
Existing model training simulators fail to support heterogeneous GPU architectures and various intra-server high-speed interconnect protocols (e.g., NVLink, HCCS), while suffering from low efficiency in full-packet-level network simulation. In this paper, we present LST-Sim, an efficient simulator designed for large-scale model training, balancing simulation accuracy and runtime efficiency. First, the simulator takes into account the impacts of computational heterogeneity across different GPU models and inter-GPU interconnection bandwidth on training latency, and constructs a theoretical model. Furthermore, we propose a domain-based traffic management mechanism that integrates theoretical analysis with network simulation to handle different types of communication traffic, thereby improving simulation efficiency. The effectiveness of LST-Sim is well validated on the Ascend 910B training cluster. Its simulation results are in close agreement with real-world measurements. Meanwhile, the domain-based traffic management mechanism significantly reduces simulation runtime by approximately 75% while maintaining high fidelity.Keywords
Funding Information
- National Key R&D Program of China (2024YFB2908701)
This publication has 9 references indexed in Scilit:
- HiCCL: A Hierarchical Collective Communication Library2025
- xCCL: A Survey of Industry-Led Collective Communication Libraries for Deep LearningJournal of Computer Science and Technology, 2023
- DeepSpeed2020
- ASTRA-SIM: Enabling SW/HW Co-Design Exploration for Distributed DL Training Platforms2020
- Accel-Sim: An Extensible Simulation Framework for Validated GPU Modeling2020
- PipeDream2019
- A Practical Introduction to the OMNeT++ Simulation Framework2019
- The ns-3 Network Simulator2010
- Analyzing CUDA workloads using a detailed GPU simulator2009