Hardware-Algorithm Co-design Enabling Processing-in-Pixel-in-Memory (P2M) for Neuromorphic Vision Sensors
Preprint
Abstract
The high volume of data transmission between the edge sensor and the cloud processor leads to energy and throughput bottlenecks for resource-constrained edge devices focused on computer vision. Hence, researchers are investigating different approaches (e.g., near-sensor processing, in-sensor processing, in-pixel processing) by executing computations closer to the sensor to reduce the transmission bandwidth. Specifically, in-pixel processing for neuromorphic vision sensors (e.g., dynamic vision sensors (DVS)) involves incorporating asynchronous multiply-accumulate (MAC) operations within the pixel array, resulting in improved energy efficiency. In a CMOS implementation, low overhead energy-efficient analog MAC accumulates charges on a passive capacitor; however, the capacitor's limited charge retention time affects the algorithmic integration time choices, impacting the algorithmic accuracy, bandwidth, energy, and training efficiency. Consequently, this results in a design trade-off on the hardware aspect-creating a need for a low-leakage compute unit while maintaining the area and energy benefits. In this work, we present a holistic analysis of the hardware-algorithm co-design trade-off based on the limited integration time posed by the hardware and techniques to improve the leakage performance of the in-pixel analog MAC operations.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- In-Sensor & Neuromorphic Computing Are all You Need for Energy Efficient Computer Vision2023
- P2M-DeTrack: Processing-in-Pixel-in-Memory for Energy-efficient and Real-Time Multi-Object Detection and Tracking2022
- A processing-in-pixel-in-memory paradigm for resource-constrained TinyML applicationsScientific Reports, 2022
- ACE-SNN: Algorithm-Hardware Co-design of Energy-Efficient & Low-Latency Deep Spiking Neural Networks for 3D Image RecognitionFrontiers in Neuroscience, 2022
- Training Energy-Efficient Deep Spiking Neural Networks with Single-Spike Hybrid Input Encoding2021
- Event-Based Neuromorphic Vision for Autonomous Driving: A Paradigm Shift for Bio-Inspired Visual Sensing and PerceptionIEEE Signal Processing Magazine, 2020
- In-sensor computing for machine visionNature, 2020
- Converting Static Image Datasets to Spiking Neuromorphic Datasets Using SaccadesFrontiers in Neuroscience, 2015
- A 3.6 $\mu$s Latency Asynchronous Frame-Free Event-Driven Dynamic-Vision-SensorIEEE Journal of Solid-State Circuits, 2011
- A 128$\times$128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision SensorIEEE Journal of Solid-State Circuits, 2008