Zhenyu Pan
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Zhenyu Pan has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Ang Li
Pacific Northwest National Laboratory
1 shared publication - Anqi Guo
Department of Electrical & Computer Engineering, Boston University, Boston, Massachusetts, USA
1 shared publication - Chunshu Wu
Pacific Northwest National Laboratory
1 shared publication - Dingwen Tao
Chinese Academy of Sciences
1 shared publication - Hui Jiang
York University, Toronto
1 shared publication - Martin Herbordt
Boston University, Boston, Massachusetts, USA
1 shared publication
Latest Publications
- Open Access
- 21 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 336-347
Software-Hardware Co-design of Heterogeneous SmartNIC System for Recommendation Models Inference and Training
Deep Learning Recommendation Models (DLRMs) are important applications in various domains and have evolved into one of the largest and most important machine learning applications. With their trillions of parameters necessarily exceeding the high bandwidth memory (HBM) capacity of GPUs, ever more massive DLRMs require large-scale multi-node systems for distributed training and inference. However, these all suffer from the all-to-all communication bottleneck, which limits scalability. SmartNICs couple computation and communication capabilities to provide powerful network-facing heterogeneous devices that reduce communication overhead. There has not, however, been a distributed system design that fully leverages SmartNIC resources to address scalability of DLRMs. We propose a software-hardware co-design of a heterogeneous SmartNIC system that overcomes the communication bottleneck of distributed DLRMs, mitigates the pressure on memory bandwidth, and improves computation efficiency. We provide a set of SmartNIC designs of cache systems (including local cache and remote cache) and SmartNIC computation kernels that reduce data movement, relieve memory lookup intensity, and improve the GPU's computation efficiency. In addition, we propose a graph algorithm that improves the data locality of queries within batches and optimizes the overall system performance with higher data reuse. Our evaluation shows that the system achieves 2.1× latency speedup for inference and 1.6× throughput speedup for training.
- 21 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 336-347
Open AccessSoftware-Hardware Co-design of Heterogeneous SmartNIC System for Recommendation Models Inference and Training
Deep Learning Recommendation Models (DLRMs) are important applications in various domains and have evolved into one of the largest and most important machine learning applications. With their trillions of parameters necessarily exceeding the high bandwidth memory (HBM) capacity of GPUs, ever more massive DLRMs require large-scale multi-node systems for distributed training and inference. However, these all suffer from the all-to-all communication bottleneck, which limits scalability. SmartNICs couple computation and communication capabilities to provide powerful network-facing heterogeneous devices that reduce communication overhead. There has not, however, been a distributed system design that fully leverages SmartNIC resources to address scalability of DLRMs. We propose a software-hardware co-design of a heterogeneous SmartNIC system that overcomes the communication bottleneck of distributed DLRMs, mitigates the pressure on memory bandwidth, and improves computation efficiency. We provide a set of SmartNIC designs of cache systems (including local cache and remote cache) and SmartNIC computation kernels that reduce data movement, relieve memory lookup intensity, and improve the GPU's computation efficiency. In addition, we propose a graph algorithm that improves the data locality of queries within batches and optimizes the overall system performance with higher data reuse. Our evaluation shows that the system achieves 2.1× latency speedup for inference and 1.6× throughput speedup for training.
- 6 June 2014
- Published by Springer Nature in Pattern Analysis and Applications
- Vol. 18 (3) , 535-551
Discriminative learning of generative models: large margin multinomial mixture models for document classification
- 6 June 2014
- Published by Springer Nature in Pattern Analysis and Applications
- Vol. 18 (3) , 535-551
Discriminative learning of generative models: large margin multinomial mixture models for document classification