Peng Xie
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Peng Xie has a total of 5 co-authors. Here are the top co-authors by number of shared publications:
- Andrew Rittenbach
Information Sciences Institute, University of Southern California, Arlington, VA, USA
1 shared publication - Connor Imes
Information Sciences Institute, University of Southern California, Arlington, VA, USA
1 shared publication - Dong In D. Kang
Information Sciences Institute, University of Southern California, Arlington, VA, USA
1 shared publication - John Paul Walters
Information Sciences Institute, University of Southern California, Arlington, VA, USA
1 shared publication - Stephen P. Crago
Information Sciences Institute, University of Southern California, Arlington, VA, USA and Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California
1 shared publication
Latest Publications
- Open Access
- 22 April 2024
- Published by Association for Computing Machinery (ACM)
- p. 169-175
Evaluating Deep Learning Recommendation Model Training Scalability with the Dynamic Opera Network
Deep learning is commonly used to make personalized recommendations to users for a wide variety of activities. However, deep learning recommendation model (DLRM) training is increasingly dominated by all-to-all and many-to-many communication patterns. While there are a wide variety of algorithms to efficiently overlap communication and computation for many collective operations, these patterns are strictly limited by network bottlenecks. We propose co-designing DLRM model training with the recently proposed Opera network, which is designed to avoid multiple network hops using time-varying source-to-destination circuits. Using measurements from state-of-the-art NVIDIA A100 GPUs, we simulate DLRM model training on networks ranging from 16 to 1024 nodes and demonstrate up to 1.79× improvement using Opera compared with equivalent fat-tree networks. We identify important parameters affecting training time and demonstrate that careful co-design is needed to optimize training latency.
- 22 April 2024
- Published by Association for Computing Machinery (ACM)
- p. 169-175
Open AccessEvaluating Deep Learning Recommendation Model Training Scalability with the Dynamic Opera Network
Deep learning is commonly used to make personalized recommendations to users for a wide variety of activities. However, deep learning recommendation model (DLRM) training is increasingly dominated by all-to-all and many-to-many communication patterns. While there are a wide variety of algorithms to efficiently overlap communication and computation for many collective operations, these patterns are strictly limited by network bottlenecks. We propose co-designing DLRM model training with the recently proposed Opera network, which is designed to avoid multiple network hops using time-varying source-to-destination circuits. Using measurements from state-of-the-art NVIDIA A100 GPUs, we simulate DLRM model training on networks ranging from 16 to 1024 nodes and demonstrate up to 1.79× improvement using Opera compared with equivalent fat-tree networks. We identify important parameters affecting training time and demonstrate that careful co-design is needed to optimize training latency.