A General-Purpose Computing Core With Cooperative Motion Detection and Feature Extraction for Always-On PWM Image Sensors
- 25 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
- Vol. 33 (10) , 2728-2739
Abstract
This article presents a mixed-signal general-purpose computing core designed to support real-time inference applications in always-on low-power pulsewidth modulation (PWM) CMOS image sensors (CISs), functioning as a processing-in-sensor (PIS) circuit. This core can be integrated into the columns of CISs to perform low-power edge processing on images, without affecting the pixel fill factor and imaging quality. It employs a coordinated mechanism in which motion detection (MD) triggers the activation of feature extraction (FE), thereby achieving an organic integration of MD and FE functionalities, and maximizing the system’s power efficiency. MD is implemented via in-column frame difference (FD), while FE is performed using a programmable-weight $3\times 3$ convolution, a rectified linear unit activation function, and a $2\times 2$ max-pooling (MP) operation. Both functionalities are computed based on real-time PWM signals from the CIS and the principle of current integration, with partial circuit reuse achieved through different switching operations. A 0.8-V computing core prototype, with an area of 720 × 272 $\mu$ m, was fabricated and verified using 0.18- $\mu $ m standard CMOS technology. The experimental results at an image frame rate of 250 fps demonstrated an average power consumption of $5.23~\mu $ W for MD and $17.53~\mu $ W for FE. The prototype core computes the first two layers of an ultra lightweight convolutional neural network (CNN) for the task of MNIST digit classification, achieving an accuracy loss of only 0.86% compared to the ideal scenario. This analog computing core can be used in multimode, low-power, edge-intelligent vision sensors.
Keywords
Funding Information
- Beijing Major Science and Technology Project (Z241100004224003)
This publication has 30 references indexed in Scilit:
- A Dual-Mode CMOS Image Sensor Based on in-Pixel Frame Differencing2024
- A 0.57 mW@1 FPS In-Column Analog CNN Processor Integrated Into CMOS Image SensorIEEE Access, 2023
- A Real-Time Edge-Detection CMOS Image Sensor for Machine Vision ApplicationsIEEE Sensors Journal, 2023
- Low-Power Techniques on a CMOS Vision Sensor Chip for Event Generation by Frame Differencing with High Dynamic Range2022
- A Weighted Current Summation Based Mixed Signal DRAM-PIM Architecture for Deep Neural Network InferenceIEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2022
- ARCHON: A 332.7TOPS/W 5b Variation-Tolerant Analog CNN Processor Featuring Analog Neuronal Computation Unit and Analog Memory2022
- A Fully Digital Time-Mode CMOS Image Sensor with 22.9pJ/frame.pixel and 92dB Dynamic Range2022
- 7.7 A 0.2-to-3.6TOPS/W Programmable Convolutional Imager SoC with In-Sensor Current-Domain Ternary-Weighted MAC Operations for Feature Extraction and Region-of-Interest Detection2021
- 9.6 A 1/2.3inch 12.3Mpixel with On-Chip 4.97TOPS/W CNN Processor Back-Illuminated Stacked CMOS Image Sensor2021
- A 467nW CMOS visual motion sensor with temporal averaging and pixel aggregation2013