Understanding Training Efficiency of Deep Learning Recommendation Models at Scale
Authors:
- Bilge AcunBABilge Acun
- Matthew MurphyMMMatthew Murphy
- Xiaodong WangXWXiaodong Wang
- Jade NieJNJade Nie
- Carole-Jean WuCWCarole-Jean Wu
- Kim HazelwoodKHKim Hazelwood
Preprint
Abstract
The use of GPUs has proliferated for machine learning workflows and is now considered mainstream for many deep learning models. Meanwhile, when training state-of-the-art personal recommendation models, which consume the highest number of compute cycles at our large-scale datacenters, the use of GPUs came with various challenges due to having both compute-intensive and memory-intensive components. GPU performance and efficiency of these recommendation models are largely affected by model architecture configurations such as dense and sparse features, MLP dimensions. Furthermore, these models often contain large embedding tables that do not fit into limited GPU memory. The goal of this paper is to explain the intricacies of using GPUs for training recommendation models, factors affecting hardware efficiency at scale, and learnings from a new scale-up GPU server design, Zion.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems2021
- Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction2020
- DeepRecSys: A System for Optimizing End-To-End At-Scale Neural Recommendation Inference2020
- MLPerf Inference Benchmark2020
- MLPerf: An Industry Standard Benchmark Suite for Machine Learning PerformanceIEEE Micro, 2020
- Billion-Scale Similarity Search with GPUsIEEE Transactions on Big Data, 2019
- FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search2019
- Deep Interest Network for Click-Through Rate Prediction2018
- Focal Loss for Dense Object Detection2017
- The tail at scaleCommunications of the ACM, 2013