A 10 000-Inference/s Bio-Inspired Spiking Vision Chip Based on an End-to-End SNN Embedding Image Signal Enhancement
- 8 July 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Solid-State Circuits
- Vol. 61 (3) , 1164-1180
Abstract
Edge machine vision requires a vision chip to perform imaging and visual recognition with high speed, high accuracy, and compact size. Current vision chips adopt a two-stage workflow, which uses computer vision processing and a deep neural network (DNN) to perform signal enhancement and recognition, respectively. However, the two stages involve different operations and are separately optimized, which causes degraded recognition accuracy, low inference rate, and huge hardware overhead. To address it, we propose a vision chip that integrates a single-photon avalanche diode (SPAD) image sensor and a reconfigurable spiking neural network (SNN) processor. Inspired by the human visual system that employs a full SNN for signal enhancement and recognition, the vision chip acquires spike-based images and performs an end-to-end SNN embedding image signal enhancement (ISE) for object recognition. The ISE and recognition are trained jointly in a unified deep learning framework to increase recognition accuracy and inference speed. The vision chip only integrates an SNN processor to perform ISE and recognition by being reconfigured into the low-precision multiply-accumulation (MAC) mode or accumulation (ACC) mode, respectively, thereby avoiding a dedicated image signal processor (ISP). Based on a 55-nm CMOS technology, the fabricated vision chip integrates $128{\times }128$ pixels and can implement 4096 ACC/MAC components. The chip was measured to exhibit: 1) a 99.2% accuracy with 10000 inferences/s at 10 lx and 95.3% at 1.43 mlx on the MNIST dataset and 2) a 94.32% accuracy with color mosaic images on the ETH-80 dataset. Our chip maintains a relatively high recognition performance even under an illumination one order of magnitude lower than in prior vision chips.Keywords
Funding Information
- National Natural Science Foundation of China (62334008)
- National Natural Science Foundation of China (62404218)
- Beijing Natural Science Foundation (Z220005)
- Beijing Municipal Science and Technology Project (Z221100007722028)
- Fundamental Research Funds for the Central Universities (2024CDJXY020)
This publication has 38 references indexed in Scilit:
- A visual cortex-inspired edge neuromorphic hardware architecture with on-chip multi-layer STDP learningComputers and Electrical Engineering, 2024
- Spike-based dynamic computing with asynchronous sensing-computing neuromorphic chipNature Communications, 2024
- Deep multi-threshold spiking-UNet for image processingNeurocomputing, 2024
- Trainable Spiking-YOLO for low-latency and high-performance object detectionNeural Networks, 2023
- Learning to See in the Dark with Events2020
- Near-sensor and in-sensor computingNature Electronics, 2020
- Towards spike-based machine intelligence with neuromorphic computingNature, 2019
- STDP-based spiking deep convolutional neural networks for object recognitionNeural Networks, 2018
- Spiking Deep Convolutional Neural Networks for Energy-Efficient Object RecognitionInternational Journal of Computer Vision, 2014
- The Neuronal Organization of the RetinaNeuron, 2012