KERNTROL: Kernel Shape Control Toward Ultimate Memory Utilization for In-Memory Convolutional Weight Mapping
- 22 February 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems I: Regular Papers
- Vol. 71 (12) , 6138-6151
Abstract
Processing-in-memory (PIM) architectures have been highlighted as one of the most viable options for faster and more power-efficient computation. Paired with a convolutional weight mapping scheme, PIM arrays can accelerate various deep convolutional neural networks (CNNs) and the applications that adopt them. Recently, shift and duplicate kernel (SDK) convolutional weight mapping scheme was proposed, achieving up to 50% throughput improvement over the prior arts. However, the traditional pattern-based pruning methods, which were adopted for row-skipping and computing cycle reduction, are not optimal for the latest SDK mapping due to the loss of structural regularity caused by the shifted and duplicated kernels. To address this challenge, we propose kernel shape control (KERNTROL), a method where kernel shapes are controlled depending on their mapped columns with the purpose of fostering a structural regularity that is favorable in achieving a high row-skipping ratio and model accuracy. Instead of permanently pruning the weights, KERNTROL with an empty mask (KERNTROL-M) temporarily omits them in the underutilized row using a utilization threshold, thereby preserving important weight elements. However, a significant portion of the memory cells is still underutilized where the threshold is not enforced. To overcome this, we extend KERNTROL-M into KERNTROL with compensatory weights (KERNTROL-C). By populating idle cells with compensatory weights, KERNTROL-C can offset the accuracy drop from weight omission. In comparison to pattern-based pruning approaches, KERNTROL-C achieves simultaneous improvements of up to 36.4% improvement in the compression rate and 5% in model accuracy with up to 100% array utilization.Keywords
Funding Information
- Technology Innovation program funded by the Ministry of Trade, Industry and Energy (RS-2023-00235718, 1415187474)
- Korea Institute of Energy Technology Evaluation and Planning (KETEP) funded by MOTIE (20224000000440)
- Institute of Information and Communication Technology Planning and Evaluation
- Ministry of Science and ICT (MSIT) of Korean Government (IITP-2021-0-02068)
This publication has 34 references indexed in Scilit:
- Kernel Shape Control for Row-Efficient Convolution on Processing-In-Memory Arrays2023
- A memristor-based Bayesian machineNature Electronics, 2022
- Activity-difference training of deep neural networks using memristor crossbarsNature Electronics, 2022
- VWC-SDK: Convolutional Weight Mapping Using Shifted and Duplicated Kernel With Variable Windows and ChannelsIEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2022
- A crossbar array of magnetoresistive memory devices for in-memory computingNature, 2022
- High PE Utilization CNN Accelerator with Channel Fusion Supporting Pattern-Compressed Sparse Neural Networks2020
- Timely: Pushing Data Movements And Interfaces In Pim Accelerators Towards Local And In Time Domain2020
- GraphR: Accelerating Graph Processing Using ReRAM2018
- PipeLayer: A Pipelined ReRAM-Based Accelerator for Deep Learning2017
- Deep Residual Learning for Image Recognition2016