Qinghao Hu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Qinghao Hu has a total of 34 co-authors. Here are the top co-authors by number of shared publications:
- Ana Klimovic
ETH Zurich, Zurich, Switzerland
3 shared publications - Peng Sun
Shanghai AI Lab, Shanghai, China
3 shared publications - Song Han
Massachusetts Institute of Technology
2 shared publications - Tiancheng Chen
ETH Zurich, Zurich, Switzerland
2 shared publications - Tianwei Zhang
Nanyang technological university, Singapore, Singapore
2 shared publications - Xiaozhe Yao
ETH Zurich, Zurich, Switzerland
2 shared publications
Latest Publications
- Open Access
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1879-1893
Zeppelin: Balancing Variable-length Workloads in Data Parallel Large Model Training
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1879-1893
Open AccessZeppelin: Balancing Variable-length Workloads in Data Parallel Large Model Training
- Open Access
- 22 March 2026
- Published by Association for Computing Machinery (ACM)
- p. 1933-1948
Taming the Long-Tail: Efficient Reasoning RL Training with Adaptive Drafter
- 22 March 2026
- Published by Association for Computing Machinery (ACM)
- p. 1933-1948
Open AccessTaming the Long-Tail: Efficient Reasoning RL Training with Adaptive Drafter
- Open Access
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 204-220
Sailor: Automating Distributed Training over Dynamic, Heterogeneous, and Geo-distributed Clusters
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 204-220
Open AccessSailor: Automating Distributed Training over Dynamic, Heterogeneous, and Geo-distributed Clusters
- Open Access
- 30 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 110-127
DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs
- 30 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 110-127
Open AccessDeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs
- Open Access
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 2902-2913
FedDSE: Distribution-aware Sub-model Extraction for Federated Learning over Resource-constrained Devices
Sub-model extraction based federated learning has emerged as a popular strategy for training models on resource-constrained devices. However, existing methods treat all clients equally and extract sub-models using predetermined rules, which disregard the statistical heterogeneity across clients and may lead to fierce competition among them. Specifically, this paper identifies that when making predictions, different clients tend to activate different neurons of the entire model related to their respective distributions. If highly activated neurons from some clients with one distribution are incorporated into the sub-model allocated to other clients with different distributions, they will be forced to fit the new distributions, which can hinder their activation over the previous clients and result in a performance reduction. Motivated by this finding, we propose a novel method called FedDSE, which can reduce the conflicts among clients by extracting sub-models based on the data distribution of each client. The core idea of FedDSE is to empower each client to adaptively extract neurons from the entire model based on their activation over the local dataset. We theoretically show that FedDSE can achieve an improved classification score and convergence over general neural networks with the ReLU activation function. Experimental results on various datasets and models show that FedDSE outperforms all state-of-the-art baselines.
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 2902-2913
Open AccessFedDSE: Distribution-aware Sub-model Extraction for Federated Learning over Resource-constrained Devices
Sub-model extraction based federated learning has emerged as a popular strategy for training models on resource-constrained devices. However, existing methods treat all clients equally and extract sub-models using predetermined rules, which disregard the statistical heterogeneity across clients and may lead to fierce competition among them. Specifically, this paper identifies that when making predictions, different clients tend to activate different neurons of the entire model related to their respective distributions. If highly activated neurons from some clients with one distribution are incorporated into the sub-model allocated to other clients with different distributions, they will be forced to fit the new distributions, which can hinder their activation over the previous clients and result in a performance reduction. Motivated by this finding, we propose a novel method called FedDSE, which can reduce the conflicts among clients by extracting sub-models based on the data distribution of each client. The core idea of FedDSE is to empower each client to adaptively extract neurons from the entire model based on their activation over the local dataset. We theoretically show that FedDSE can achieve an improved classification score and convergence over general neural networks with the ReLU activation function. Experimental results on various datasets and models show that FedDSE outperforms all state-of-the-art baselines.
- Top Cited PapersOpen Access
- 22 January 2024
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
- Vol. 56 (6) , 1-38
Deep Learning Workload Scheduling in GPU Datacenters: A Survey
Deep learning (DL) has demonstrated its remarkable success in a wide variety of fields. The development of a DL model is a time-consuming and resource-intensive procedure. Hence, dedicated GPU accelerators have been collectively constructed into a GPU datacenter. An efficient scheduler design for a GPU datacenter is crucially important to reduce operational cost and improve resource utilization. However, traditional approaches designed for big data or high-performance computing workloads can not support DL workloads to fully utilize the GPU resources. Recently, many schedulers are proposed to tailor for DL workloads in GPU datacenters. This article surveys existing research efforts for both training and inference workloads. We primarily present how existing schedulers facilitate the respective workloads from the scheduling objectives and resource utilization manner . Finally, we discuss several promising future research directions including emerging DL workloads, advanced scheduling decision making, and underlying hardware resources. A more detailed summary of the surveyed paper and code links can be found at our project website: https://github.com/S-Lab-System-Group/Awesome-DL-Scheduling-Papers
- 22 January 2024
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
- Vol. 56 (6) , 1-38
Top Cited PapersOpen AccessDeep Learning Workload Scheduling in GPU Datacenters: A Survey
Deep learning (DL) has demonstrated its remarkable success in a wide variety of fields. The development of a DL model is a time-consuming and resource-intensive procedure. Hence, dedicated GPU accelerators have been collectively constructed into a GPU datacenter. An efficient scheduler design for a GPU datacenter is crucially important to reduce operational cost and improve resource utilization. However, traditional approaches designed for big data or high-performance computing workloads can not support DL workloads to fully utilize the GPU resources. Recently, many schedulers are proposed to tailor for DL workloads in GPU datacenters. This article surveys existing research efforts for both training and inference workloads. We primarily present how existing schedulers facilitate the respective workloads from the scheduling objectives and resource utilization manner . Finally, we discuss several promising future research directions including emerging DL workloads, advanced scheduling decision making, and underlying hardware resources. A more detailed summary of the surveyed paper and code links can be found at our project website: https://github.com/S-Lab-System-Group/Awesome-DL-Scheduling-Papers