Runtime Support for Accelerating CNN Models on Digital DRAM Processing-in-Memory Hardware
- 13 June 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 21 (2) , 33-36
Abstract
Processing-in-memory (PIM) provides promising solutions to the main memory bottleneck by placing computational logic in or near memory devices to reduce data movement overheads. Recent work explored how commercial DRAM can feature digital PIM logic while meeting fab-level energy and area constraints, and showed a significant speedup in the inference time of data-intensive deep learning models. However, convolutional neural network (CNN) models were not considered as main targets for the commercial DRAM-PIM due to their compute-intensive convolution layers. Moreover, recent studies revealed that the area and power constraints on memory die prevent DRAM-PIM from competing with GPUs and specialized accelerators in accelerating them. Recently, mobile CNN models have increasingly adopted a composition of depthwise and pointwise convolutions instead of such compute-intensive convolutions to reduce computation cost without accuracy drop. In this paper, we show that 1x1 convolution can be offloaded for PIM acceleration with integrated runtime support and without any hardware or algorithm changes. We provide further speedup with parallel execution on GPU and DRAM-PIM and code generation optimizations. Our solution achieves up to 35.2% (31.6% on average) speedup for all 1x1 convolutions for mobile CNN models against GPU.Keywords
Funding Information
- Institute for Information and Communications Technology Promotion (2019-0-01906, 2021-0-00871, 2021-0-00310)
This publication has 14 references indexed in Scilit:
- Accel-Sim: An Extensible Simulation Framework for Validated GPU Modeling2020
- RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing2020
- MnasNet: Platform-Aware Neural Architecture Search for Mobile2019
- MobileNetV2: Inverted Residuals and Linear Bottlenecks2018
- In-Datacenter Performance Analysis of a Tensor Processing UnitACM SIGARCH Computer Architecture News, 2017
- Deep Residual Learning for Image Recognition2016
- Ramulator: A Fast and Extensible DRAM SimulatorIEEE Computer Architecture Letters, 2015
- The architecture of the DIVA processing-in-memory chip2002
- Long Short-Term MemoryNeural Computation, 1997
- Processing in memory: the Terasys massively parallel PIM arrayComputer, 1995