Li Li
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Li Li has a total of 36 co-authors. Here are the top co-authors by number of shared publications:
- Minyi Guo
Shanghai Jiao Tong University, Shanghai, China
7 shared publications - Quan Chen
Shanghai Jiao Tong University
4 shared publications - Pu Pang
Shanghai Jiao Tong University, Shanghai, China
3 shared publications - Jingwen Leng
Shanghai Jiao Tong University, Shanghai, China
2 shared publications - Kun Wang
Fudan University, Shanghai, Shanghai, China
2 shared publications - Song Guo
Hong Kong University of Science and Technology, Hong Kong, China
2 shared publications
Latest Publications
- Open Access
- 27 September 2026
- Published by Association for Computing Machinery (ACM)
- p. 45-55
Aethon: Performance-aware Memory Offloading for Co-running Applications in Public Clouds
- 27 September 2026
- Published by Association for Computing Machinery (ACM)
- p. 45-55
Open AccessAethon: Performance-aware Memory Offloading for Co-running Applications in Public Clouds
- 16 November 2025
- Published by Springer Nature
- p. 448-460
Reducing Load-Balancing Cost for Multithreading Applications on Asymmetric NUMA Machine
- 16 November 2025
- Published by Springer Nature
- p. 448-460
Reducing Load-Balancing Cost for Multithreading Applications on Asymmetric NUMA Machine
- Open Access
- 8 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 895-910
Generating Microservice Graphs with Production Characteristics for Efficient Resource Scaling
- 8 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 895-910
Open AccessGenerating Microservice Graphs with Production Characteristics for Efficient Resource Scaling
- Open Access
- 9 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 52-62
AdaptGear
Graph neural networks (GNNs) are powerful tools for exploring and learning from graph structures and features. As such, achieving high-performance execution for GNNs becomes crucially important. Prior works have proposed to explore the sparsity (i.e., low density) in the input graph to accelerate GNNs, which uses the full-graph-level or block-level sparsity format. We show that they fail to balance the sparsity benefit and kernel execution efficiency. In this paper, we propose a novel system, referred to as AdaptGear, that addresses the challenge of optimizing GNNs performance by leveraging kernels tailored to the density characteristics at the subgraph level. Meanwhile, we also propose a method that dynamically chooses the optimal set of kernels for a given input graph. Our evaluation shows that AdaptGear can achieve a significant performance improvement, up to 6.49× (1.87× on average), over the state-of-the-art works on two mainstream NVIDIA GPUs across various datasets.
- 9 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 52-62
Open AccessAdaptGear
Graph neural networks (GNNs) are powerful tools for exploring and learning from graph structures and features. As such, achieving high-performance execution for GNNs becomes crucially important. Prior works have proposed to explore the sparsity (i.e., low density) in the input graph to accelerate GNNs, which uses the full-graph-level or block-level sparsity format. We show that they fail to balance the sparsity benefit and kernel execution efficiency. In this paper, we propose a novel system, referred to as AdaptGear, that addresses the challenge of optimizing GNNs performance by leveraging kernels tailored to the density characteristics at the subgraph level. Meanwhile, we also propose a method that dynamically chooses the optimal set of kernels for a given input graph. Our evaluation shows that AdaptGear can achieve a significant performance improvement, up to 6.49× (1.87× on average), over the state-of-the-art works on two mainstream NVIDIA GPUs across various datasets.
- Open Access
- 9 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 112-122
DistSim
With the ever-increasing computational demand of DNN training workloads, distributed training has been widely adopted. A combination of data, model and pipeline parallelism strategy, called hybrid parallelism distributed training, is imported to tackle the problem of deploying large-scale models. However, how to evaluate the hybrid strategy and the utilization of each device remains a challenge since existing works either profile on a real large-scale cluster with high time and money costs or only analyze a specific type of parallelism without considering the hybrid parallelism. In this work, we proposed DistSim, an event-based performance model to accurately analyze each device's computation and communication activities with low profiling costs. DistDim breaks down the model into events according to the given distributed strategy, which can be profiled on two nodes. Then DistSim leverages the hierarchy of different parallel strategies to generate the computation and communication event-flow from layer level to model level and finally the activity timeline of each device participating in training. Experiment shows that DistSim can reach <4% errors when predicting distributing training batch time and <5% errors when predicting a single device's activity time in various hybrid strategy settings. We also provide a use-case of DistSim, automatically evaluate and search the best distributed training strategy, and find a hybrid strategy with at most 7.37× throughput improvement.
- 9 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 112-122
Open AccessDistSim
With the ever-increasing computational demand of DNN training workloads, distributed training has been widely adopted. A combination of data, model and pipeline parallelism strategy, called hybrid parallelism distributed training, is imported to tackle the problem of deploying large-scale models. However, how to evaluate the hybrid strategy and the utilization of each device remains a challenge since existing works either profile on a real large-scale cluster with high time and money costs or only analyze a specific type of parallelism without considering the hybrid parallelism. In this work, we proposed DistSim, an event-based performance model to accurately analyze each device's computation and communication activities with low profiling costs. DistDim breaks down the model into events according to the given distributed strategy, which can be profiled on two nodes. Then DistSim leverages the hierarchy of different parallel strategies to generate the computation and communication event-flow from layer level to model level and finally the activity timeline of each device participating in training. Experiment shows that DistSim can reach <4% errors when predicting distributing training batch time and <5% errors when predicting a single device's activity time in various hybrid strategy settings. We also provide a use-case of DistSim, automatically evaluate and search the best distributed training strategy, and find a hybrid strategy with at most 7.37× throughput improvement.
- 24 September 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in Computer
- Vol. 54 (10) , 4-5
A Comprehensive Inspection of the Straggler Problem
- 24 September 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in Computer
- Vol. 54 (10) , 4-5
A Comprehensive Inspection of the Straggler Problem
- 25 November 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 32 (5) , 1030-1043
Petrel: Heterogeneity-Aware Distributed Deep Learning Via Hybrid Synchronization
The parameter server (PS) paradigm has achieved great success in deploying large-scale distributed Deep Learning (DL) systems. However, the implicit assumption of homogeneous clusters in these systems does not hold in many real-world cases. Also, existing heterogeneity-aware approaches mainly prioritize the contribution of fast workers and reduce the involvement of slow workers, resulting in the limitations of workload imbalance and computation inefficiency. We reveal that grouping workers into communities, an abstraction proposed by us, and handling parameter synchronization in community level can conquer these limitations and accelerate training convergence progress. The inspiration of community comes from our exploration of prior knowledge about the similarity between workers, which is often neglected by previous work. These observations motivate us to propose a new synchronization mechanism named Community-aware Synchronous Parallel (CASP), which uses the Asynchronous Advantage Actor-Critic (A3C) to intelligently determine community configuration. The whole idea has been implemented in a system called Petrel that achieves a good balance between convergence efficiency and communication overhead. Evaluation under various benchmarks demonstrates the superiority of Petrel. Specifically, Petrel accelerates the training convergence speed by up to 1.87 times faster and reduces communication traffic by up to 26.85%, on average, over the non-community synchronization mechanisms.
- 25 November 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 32 (5) , 1030-1043
Petrel: Heterogeneity-Aware Distributed Deep Learning Via Hybrid Synchronization
The parameter server (PS) paradigm has achieved great success in deploying large-scale distributed Deep Learning (DL) systems. However, the implicit assumption of homogeneous clusters in these systems does not hold in many real-world cases. Also, existing heterogeneity-aware approaches mainly prioritize the contribution of fast workers and reduce the involvement of slow workers, resulting in the limitations of workload imbalance and computation inefficiency. We reveal that grouping workers into communities, an abstraction proposed by us, and handling parameter synchronization in community level can conquer these limitations and accelerate training convergence progress. The inspiration of community comes from our exploration of prior knowledge about the similarity between workers, which is often neglected by previous work. These observations motivate us to propose a new synchronization mechanism named Community-aware Synchronous Parallel (CASP), which uses the Asynchronous Advantage Actor-Critic (A3C) to intelligently determine community configuration. The whole idea has been implemented in a system called Petrel that achieves a good balance between convergence efficiency and communication overhead. Evaluation under various benchmarks demonstrates the superiority of Petrel. Specifically, Petrel accelerates the training convergence speed by up to 1.87 times faster and reduces communication traffic by up to 26.85%, on average, over the non-community synchronization mechanisms.