Meng Pang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Meng Pang has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Peng Qu
Tsinghua University, Beijing, China
3 shared publications - Youhui Zhang
Tsinghua University, Beijing, China
3 shared publications - Hui Lin
College of Pharmacy, Jinan University, Guangzhou 510632, PR China
1 shared publication - Jiajie Chen
Tsinghua University, Beijing, China
1 shared publication Weimin Zheng
Department of Computer Science and Technology, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China
1 shared publication- Xiang Fei
Department of Computer Science and Technology, Tsinghua University, Beijing, China
1 shared publication
Latest Publications
- Open Access
- 20 February 2024
- Published by Association for Computing Machinery (ACM)
- p. 377-389
A Row Decomposition-based Approach for Sparse Matrix Multiplication on GPUs
Sparse-Matrix Dense-Matrix Multiplication (SpMM) and Sampled Dense Dense Matrix Multiplication (SDDMM) are important sparse kernels in various computation domains. The uneven distribution of nonzeros in the sparse matrix and the tight data dependence between sparse and dense matrices make it a challenge to run sparse matrix multiplication efficiently on GPUs. By analyzing the aforementioned problems, we propose a row decomposition (RoDe)-based approach to optimize the two kernels on GPUs, using the standard Compressed Sparse Row (CSR) format. Specifically, RoDe divides the sparse matrix rows into regular parts and residual parts, to fully optimize their computations separately. We also devise the corresponding load balancing and finegrained pipelining technologies. Profiling results show that RoDe can achieve more efficient memory access and reduce warp stall cycles significantly. Compared to the state-of-the-art (SOTA) alternatives, RoDe achieves a speedup of up to 7.86× with a geometric mean of 1.45× for SpMM, and a speedup of up to 8.99× with a geometric mean of 1.49× for SDDMM; the dataset is SuiteSparse. RoDe also outperforms its counterpart in the deep learning dataset. Furthermore, its preprocessing overhead is significantly smaller, averaging only 16% of the SOTA.
- 20 February 2024
- Published by Association for Computing Machinery (ACM)
- p. 377-389
Open AccessA Row Decomposition-based Approach for Sparse Matrix Multiplication on GPUs
Sparse-Matrix Dense-Matrix Multiplication (SpMM) and Sampled Dense Dense Matrix Multiplication (SDDMM) are important sparse kernels in various computation domains. The uneven distribution of nonzeros in the sparse matrix and the tight data dependence between sparse and dense matrices make it a challenge to run sparse matrix multiplication efficiently on GPUs. By analyzing the aforementioned problems, we propose a row decomposition (RoDe)-based approach to optimize the two kernels on GPUs, using the standard Compressed Sparse Row (CSR) format. Specifically, RoDe divides the sparse matrix rows into regular parts and residual parts, to fully optimize their computations separately. We also devise the corresponding load balancing and finegrained pipelining technologies. Profiling results show that RoDe can achieve more efficient memory access and reduce warp stall cycles significantly. Compared to the state-of-the-art (SOTA) alternatives, RoDe achieves a speedup of up to 7.86× with a geometric mean of 1.45× for SpMM, and a speedup of up to 8.99× with a geometric mean of 1.49× for SDDMM; the dataset is SuiteSparse. RoDe also outperforms its counterpart in the deep learning dataset. Furthermore, its preprocessing overhead is significantly smaller, averaging only 16% of the SOTA.
- 30 January 2024
- Published by Springer Nature in Journal of Computer Science and Technology
- Vol. 39 (1) , 4-21
Research on General-Purpose Brain-Inspired Computing Systems
- 30 January 2024
- Published by Springer Nature in Journal of Computer Science and Technology
- Vol. 39 (1) , 4-21
Research on General-Purpose Brain-Inspired Computing Systems
- 3 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 34 (9) , 2529-2540
ENLARGE: An Efficient SNN Simulation Framework on GPU Clusters
Spiking Neural Networks (SNNs) are currently the most widely used computing model for neuroscience communities. There is also an increasing research interest in exploring the potential of SNN in brain-inspired computing, artificial intelligence, and other areas. As SNNs possess distinguished characteristics that originate from biological authenticity, they require dedicated simulation frameworks to achieve usability and efficiency. However, there is no widely-used, easily accessible, high performance SNN simulation framework for GPU clusters. In this paper, we propose ENLARGE, an efficient SNN simulation framework on GPU clusters. ENLARGE provides a multi-level architecture that deals with computation, communication, and synchronization hierarchically. We also propose an efficient communication method with an all-to-all communication pattern. To deal with the delay of spike delivery, which is the most distinguished SNN characteristic, several delay-aware optimization methods are also proposed. We further propose a multilevel workload management method. Various experiments are carried out to demonstrate the performance and scalability of the framework, as well as the effects of the optimization methods. Test results show that ENLARGE can achieve$3.17\times \sim 28.12\times$ speedup compared with the most widely used NEST simulator and$3.26\times \sim 13.57\times$ speedup compared with the widely used NEST GPU simulator for GPU clusters.- 3 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 34 (9) , 2529-2540
ENLARGE: An Efficient SNN Simulation Framework on GPU Clusters
Spiking Neural Networks (SNNs) are currently the most widely used computing model for neuroscience communities. There is also an increasing research interest in exploring the potential of SNN in brain-inspired computing, artificial intelligence, and other areas. As SNNs possess distinguished characteristics that originate from biological authenticity, they require dedicated simulation frameworks to achieve usability and efficiency. However, there is no widely-used, easily accessible, high performance SNN simulation framework for GPU clusters. In this paper, we propose ENLARGE, an efficient SNN simulation framework on GPU clusters. ENLARGE provides a multi-level architecture that deals with computation, communication, and synchronization hierarchically. We also propose an efficient communication method with an all-to-all communication pattern. To deal with the delay of spike delivery, which is the most distinguished SNN characteristic, several delay-aware optimization methods are also proposed. We further propose a multilevel workload management method. Various experiments are carried out to demonstrate the performance and scalability of the framework, as well as the effects of the optimization methods. Test results show that ENLARGE can achieve$3.17\times \sim 28.12\times$ speedup compared with the most widely used NEST simulator and$3.26\times \sim 13.57\times$ speedup compared with the widely used NEST GPU simulator for GPU clusters.