Haoxuan Chen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Haoxuan Chen has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Feifei Niu
School of Electrical Engineering and Computer Science The State Key Laboratory of Blockchain University of Ottawa Ottawa Canada
1 shared publication - Fuyang Li
School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan, China
1 shared publication - Jacky Keung
City University of Hong Kong, Hong Kong, Hong Kong
1 shared publication - Lin Zhu
School of Computer, Wuhan Qingchuan University, Wuhan, China
1 shared publication - Shan Gao
Hokkaido University, Sapporo, Japan
1 shared publication - Shuang Yin
School of Computer, Wuhan Qingchuan University, Wuhan, China
1 shared publication
Latest Publications
- Open Access
Towards Understanding Bugs in Distributed Training and Inference Frameworks for Large Language Models
With the rapid development of large language models (LLMs), distributed training and inference frameworks like DeepSpeed have become essential for scaling model training and inference across multiple GPUs or nodes. However, the increasing complexity of these frameworks brings non-trivial software bugs, which may degrade training performance, cause unexpected failures, and result in significant resource waste. Understanding framework bugs' characteristics is fundamental for quality assurance, allowing the design of more effective debugging and repair methods. Thus, our paper conducts the first large-scale empirical analysis of 308 fixed bugs across three popular distributed training/inference frameworks: DeepSpeed, Megatron-LM, and Colossal-AI. We examine bug symptoms, root causes, bug identification and fixing efforts, and common low-effort fixing strategies. Additionally, the distributed nature of these frameworks introduces unique bug root causes, such as allocation strategy error and distributed communication error. Diagnosing and fixing complex bugs remains challenging due to factors like the disconnect between symptoms and root causes, high bug reproduction costs, and low-level or cross-component interactions. Interestingly, we observe that 48% of bug fixes require minimal code changes (<=10 LOC) and follow simple strategies such as conditional logic optimization, parameter handling enhancement, or version compatibility handling, indicating potential for automation. Based on these insights, we offer several implications for improving the reliability of both distributed training and inference frameworks and their dependent LLM projects, while also identifying opportunities to leverage LLM-based tools for automated debugging and repair.
Open AccessTowards Understanding Bugs in Distributed Training and Inference Frameworks for Large Language Models
With the rapid development of large language models (LLMs), distributed training and inference frameworks like DeepSpeed have become essential for scaling model training and inference across multiple GPUs or nodes. However, the increasing complexity of these frameworks brings non-trivial software bugs, which may degrade training performance, cause unexpected failures, and result in significant resource waste. Understanding framework bugs' characteristics is fundamental for quality assurance, allowing the design of more effective debugging and repair methods. Thus, our paper conducts the first large-scale empirical analysis of 308 fixed bugs across three popular distributed training/inference frameworks: DeepSpeed, Megatron-LM, and Colossal-AI. We examine bug symptoms, root causes, bug identification and fixing efforts, and common low-effort fixing strategies. Additionally, the distributed nature of these frameworks introduces unique bug root causes, such as allocation strategy error and distributed communication error. Diagnosing and fixing complex bugs remains challenging due to factors like the disconnect between symptoms and root causes, high bug reproduction costs, and low-level or cross-component interactions. Interestingly, we observe that 48% of bug fixes require minimal code changes (<=10 LOC) and follow simple strategies such as conditional logic optimization, parameter handling enhancement, or version compatibility handling, indicating potential for automation. Based on these insights, we offer several implications for improving the reliability of both distributed training and inference frameworks and their dependent LLM projects, while also identifying opportunities to leverage LLM-based tools for automated debugging and repair.
- 16 May 2025
- Published by Springer Nature in Automated Software Engineering
- Vol. 32 (2) , 1-49
The impact of feature selection and feature reduction techniques for code smell detection: A comprehensive empirical study
- 16 May 2025
- Published by Springer Nature in Automated Software Engineering
- Vol. 32 (2) , 1-49
The impact of feature selection and feature reduction techniques for code smell detection: A comprehensive empirical study