LMAI2C: Low memory access Im2col method for CNN inference
Open Access
- 10 June 2025
- journal article
- Published by Institute of Electronics, Information and Communications Engineers (IEICE) in IEICE Electronics Express
- Vol. 22 (11) , 20250246
Abstract
For neural network accelerators with General Matrix Multiplication (GEMM) as the computational core, the input feature maps of convolution must be converted into 2D matrices through the Im2col operation. Conventional approaches utilize CPUs to execute Im2col management and data transfer operations. Conventional methods suffer from memory expansion due to redundant data in overlapping convolutional windows, thus incurring non-negligible memory access energy consumption and transmission latency overheads. This severely limits the feasibility of efficient GEMM acceleration in resource-constrained edge devices. This paper proposes a novel Low Memory Access Im2col Method (LMAI2C) and present its dedicated hardware implementation. By restructuring data from overlapping convolutional windows, LMAI2C significantly reduces DRAM memory access volume while improving feature map transfer efficiency. Experimental results on convolutional layers of the YOLOv4-tiny network demonstrate that LMAI2C reduces DDR memory access by approximately 79.8% compared to traditional methods. Furthermore, LMAI2C demonstrates an average speedup of 69 times compared to CPU-based methodologies and 43 times over DMA-accelerated CPU implementations.Keywords
This publication has 30 references indexed in Scilit:
- TS-CNN: A Three-Tier Self-Interpretable CNN for Multi-Region Medical Image ClassificationIEEE Access, 2023
- Object Detection in 20 Years: A SurveyProceedings of the IEEE, 2023
- Gemmini: Enabling Systematic Deep-Learning Architecture Evaluation via Full-Stack Integration2021
- Characterizing and Demystifying the Implicit Convolution Algorithm on Commercial Matrix-Multiplication Accelerators2021
- Duplo: Lifting Redundant Memory Accesses of Deep Neural Networks for GPU Tensor Cores2020
- Deep Learning Training At Scale Spring Crest Deep Learning Accelerator (Intel® Nervana™ NNP-T)2019
- Eyeriss v2: A Flexible Accelerator for Emerging Deep Neural Networks on Mobile DevicesIEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2019
- A Single Attention-Based Combination of CNN and RNN for Relation ClassificationIEEE Access, 2019
- UNPU: An Energy-Efficient Deep Neural Network Accelerator With Fully Variable Weight Bit PrecisionIEEE Journal of Solid-State Circuits, 2018
- Deep learningNature, 2015