Heng Pan
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Heng Pan has a total of 19 co-authors. Here are the top co-authors by number of shared publications:
- Fei Feng
Alibaba Group, Hangzhou, Zhejiang, China
1 shared publication - Guohui Wang
Alibaba Group, Hangzhou, Zhejiang, China
1 shared publication - Hao Li
Alibaba Group, Hangzhou, Zhejiang, China
1 shared publication - Jianbo Dong
Alibaba Cloud, Beijing, China
1 shared publication - Jie Zhang
Alibaba Group, Hangzhou, Zhejiang, China
1 shared publication - Lingbo Tang
Alibaba Group, Hangzhou, Zhejiang, China
1 shared publication
Latest Publications
- 22 June 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Micro
- Vol. 41 (5) , 85-92
ACCL: Architecting Highly Scalable Distributed Training Systems With Highly Efficient Collective Communication Library
Distributed systems have been widely adopted for Deep Neural Networks (DNNs) model training. However, the scalability of distributed training systems is largely bounded by the communication cost. We design a highly-efficient collective communication library, namely Alibaba Collective Communication Library(ACCL), to build distributed training systems with linear scalability. ACCL provides optimized algorithms to fully make use of heterogeneous interconnects simultaneously. And the experimental results show significant performance improvement.decoupling the algorithm and communication operations. By leveraging ACCL, we propose a hybrid algorithm to fully make use of the intra- and inter-node interconnects simultaneously. The experimental results show that the communication performance of ACCL is about 3x over NCCL. And the end-to-end performance of applications can be improved by 12.6% to 25.2%, with respect to the ratio of communication time.
- 22 June 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Micro
- Vol. 41 (5) , 85-92
ACCL: Architecting Highly Scalable Distributed Training Systems With Highly Efficient Collective Communication Library
Distributed systems have been widely adopted for Deep Neural Networks (DNNs) model training. However, the scalability of distributed training systems is largely bounded by the communication cost. We design a highly-efficient collective communication library, namely Alibaba Collective Communication Library(ACCL), to build distributed training systems with linear scalability. ACCL provides optimized algorithms to fully make use of heterogeneous interconnects simultaneously. And the experimental results show significant performance improvement.decoupling the algorithm and communication operations. By leveraging ACCL, we propose a hybrid algorithm to fully make use of the intra- and inter-node interconnects simultaneously. The experimental results show that the communication performance of ACCL is about 3x over NCCL. And the end-to-end performance of applications can be improved by 12.6% to 25.2%, with respect to the ratio of communication time.