NDRec: A Near-Data Processing System for Training Large-Scale Recommendation Models
- 15 February 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 73 (5) , 1248-1261
Abstract
Recent advances in deep neural networks (DNNs) have enabled highly effective recommendation models for diverse web services. In such DNN-based recommendation models, the embedding layer comprises the majority of model parameters. As these models scale rapidly, the embedding layer’s memory capacity and bandwidth requirements threaten to exceed the limits of current computing architectures. We observe the embedding layer’s computational demands increase much more slowly than its storage needs, suggesting an opportunity to offload embeddings to storage hardware. In this work, we present NDRec, a near-data processing system to train large-scale recommendation models. NDRec offloads both the parameters and the computation of the embedding layer to computational storage devices (CSDs), using coherence interconnects (CXLs) for communication between GPUs and CSDs. By leveraging the statistical properties of embedding access patterns, we develop an optimized CSD memory hierarchy and caching strategy. A lookahead embedding scheme enables concurrent execution of embeddings and other operations, hiding latency and reducing memory bandwidth requirements.We evaluate NDRec using real-world and synthetic benchmarks. Results demonstrate NDRec achieves up to 4.33× and 3.97× speedups over heterogeneous CPU-GPU platforms and GPU caching, respectively. NDRec also reduces per-iteration energy consumption by up to 54.9%.Keywords
Funding Information
- NSF (CNS-2112562, CNS-1822085)
This publication has 29 references indexed in Scilit:
- RM-SSD: In-Storage Computing for Large-Scale Recommendation Inference2022
- Near-Memory Processing in Action: Accelerating Personalized Recommendation With AxDIMMIEEE Micro, 2021
- Understanding Training Efficiency of Deep Learning Recommendation Models at Scale2021
- SmartSSD: FPGA Accelerated Near-Storage Data Analytics on SSDIEEE Computer Architecture Letters, 2020
- RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing2020
- DRAMsim3: A Cycle-Accurate, Thermal-Capable DRAM SimulatorIEEE Computer Architecture Letters, 2020
- The Architectural Implications of Facebook's DNN-Based Personalized Recommendation2020
- TensorDIMM2019
- Multi-Rate Deep Learning for Temporal Recommendation2016
- A model of a trust-based recommendation system on a social networkAutonomous Agents and Multi-Agent Systems, 2007