TRiM: Enhancing Processor-Memory Interfaces with Scalable Tensor Reduction in Memory
Authors:
Open Access
- 17 October 2021
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 268-281
Abstract
Personalized recommendation systems are gaining significant traction due to their industrial importance. An important building block of recommendation systems consists of the embedding layers, which exhibit a highly memory-intensive characteristic. A fundamental primitive of embedding layers is the embedding vector gathers followed by vector reductions, exhibiting low arithmetic intensity and becoming bottlenecked by the memory throughput. To tackle such a challenge, recent proposals employ a near-data processing (NDP) solution at the DRAM rank-level, achieving impressive performance speedups. We observe that prior rank-level-parallelism-based NDP solutions leave significant performance potential on the table as they do not fully reap the abundant transfer throughput inherent in DRAM datapaths. We propose TRiM, an NDP architecture for accelerating recommendation systems. Based on the observation that the DRAM datapath has a hierarchical tree structure, TRiM augments the DRAM datapath with “in-DRAM” reduction units at the DDR4/5 rank/bank-group/bank level. We modify the interface of DRAM to provide commands effectively to multiple reduction units running in parallel. We also propose a host-side architecture with hot embedding-vector replication to alleviate the load imbalance that arises across the reduction units. An optimal TRiM design based on DDR5 achieves up to a 7.7 × and 3.9 × speedup and reduces by 55% and 50% the energy consumption of the embedding vector gather and reduction over the baseline and the state-of-the-art NDP architecture with minimal area overhead equivalent to 2.66% of DRAM chips.Keywords
Funding Information
- MOTIE/KEIT (10077609)
- National Research Foundation of Korea (NRF-2018R1A5A1059921)
This publication has 45 references indexed in Scilit:
- TETRIS2017
- Chameleon: Versatile and practical near-DRAM acceleration architecture for large memory systems2016
- Deep Neural Networks for YouTube Recommendations2016
- Neurocube: A Programmable Digital Neuromorphic Architecture with High-Density 3D Memory2016
- Improving DRAM latency with dynamic asymmetric subarray2015
- Ramulator: A Fast and Extensible DRAM SimulatorIEEE Computer Architecture Letters, 2015
- NDA: Near-DRAM acceleration architecture leveraging commodity DRAM devices and standard memory modules2015
- CiDRA: A cache-inspired DRAM resilience architecture2015
- A 3.2 Gbps/pin 8 Gbit 1.0 V LPDDR4 SDRAM With Integrated ECC Engine for Sub-1 V DRAM Core OperationIEEE Journal of Solid-State Circuits, 2014
- CACTI-IO2012