PIM-DL: Expanding the Applicability of Commodity DRAM-PIMs for Deep Learning via Algorithm-System Co-Optimization
Open Access
- 27 April 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 879-896
Abstract
DRAM-based processing-in-memory (DRAM-PIM) has gained commercial prominence in recent years. However, their integration for deep learning acceleration poses inherent challenges. Existing DRAM-PIMs are limited in computational capabilities, primarily applicable for element-wise and GEMV operators. Unfortunately, these operators contribute only a small portion of the execution time in most DNN workloads. Current systems still necessitate powerful hosts to handle a significant portion of compute-heavy operators. To expand the applicability of commodity DRAM-PIMs in accelerating deep learning, we introduce a novel PIM-DL framework. The philosophy behind PIM-DL is to replace the compute-heavy GEMM operations in linear layers with Lookup-Tables (LUTs). Such LUT-based neural networks (LUT-NNs) substantially reduce multiplications in DNN inference, rendering them suitable for efficient execution on DRAM-PIMs. To accurately convert DNNs into LUT-NNs and achieve optimal inference serving performance, we first introduce an enhanced LUT-NN (eLUT-NN) algorithm for model calibration, then we propose an Auto-Tuner capable of optimizing the mapping parameters on diverse DRAM-PIM platforms. We evaluate PIM-DL on off-the-shelf UPMEM PIM-DIMM products and simulated HBM-PIM/AiM platforms across multiple contemporary DNN workloads. Compared with GEMM-based inference on DRAM-PIMs, PIM-DL achieves 22.6×~37.1× speedup. Compared with CPU/GPU-based inference, PIM-DL achieves up to 3.54×/1.20× speedup.Keywords
Funding Information
- NSF China (62032001)
- 111 Project (B18001)
This publication has 66 references indexed in Scilit:
- Benchmarking a New Paradigm: Experimental Analysis and Characterization of a Real Processing-in-Memory SystemIEEE Access, 2022
- Task Parallelism-Aware Deep Neural Network Scheduling on Multiple Hybrid Memory Cube-Based Processing-in-MemoryIEEE Access, 2021
- FAFNIR: Accelerating Sparse Gathering by Using Efficient Near-Memory Intelligent Reduction2021
- SynCron: Efficient Synchronization Support for Near-Data-Processing Architectures2021
- Variant Calling Parallelization on Processor-in-Memory Architecture2020
- End-to-End Object Detection with Transformers2020
- The true Processing In Memory accelerator2019
- GraphP: Reducing Communication for PIM-Based Graph Processing with Efficient Data Partition2018
- NDA: Near-DRAM acceleration architecture leveraging commodity DRAM devices and standard memory modules2015
- A case for intelligent RAMIEEE Micro, 1997