Region Masking to Accelerate Video Processing on Neuromorphic Hardware
Preprint
Abstract
The rapidly growing demand for on-chip edge intelligence on resource-constrained devices has motivated approaches to reduce energy and latency of deep learning models. Spiking neural networks (SNNs) have gained particular interest due to their promise to reduce energy consumption using event-based processing. We assert that while sigma-delta encoding in SNNs can take advantage of the temporal redundancy across video frames, they still involve a significant amount of redundant computations due to processing insignificant events. In this paper, we propose a region masking strategy that identifies regions of interest at the input of the SNN, thereby eliminating computation and data movement for events arising from unimportant regions. Our approach demonstrates that masking regions at the input not only significantly reduces the overall spiking activity of the network, but also provides significant improvement in throughput and latency. We apply region masking during video object detection on Loihi 2, demonstrating that masking approximately 60% of input regions can reduce energy-delay product by 1.65x over a baseline sigma-delta network, with a degradation in mAP@0.5 by 1.09%.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- MaskVD: Region Masking for Efficient Video Object Detection2025
- HiRISE: High-Resolution Image Scaling for Edge ML via In-Sensor Compression and Selective ROI2024
- Beyond classification: directly training spiking neural networks for semantic segmentationNeuromorphic Computing and Engineering, 2022
- Toward Scalable, Efficient, and Accurate Deep Spiking Neural Networks With Backward Residual Connections, Stochastic Softmax, and HybridizationFrontiers in Neuroscience, 2020
- Exploring the Connection Between Binary and Spiking Neural NetworksFrontiers in Neuroscience, 2020
- A Soft-Pruning Method Applied During Training of Spiking Neural Networks for In-memory Computing ApplicationsFrontiers in Neuroscience, 2019
- STDP-Based Pruning of Connections and Weight Quantization in Spiking Neural Networks for Energy-Efficient RecognitionIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2018
- Loihi: A Neuromorphic Manycore Processor with On-Chip LearningIEEE Micro, 2018
- Spiking Deep Convolutional Neural Networks for Energy-Efficient Object RecognitionInternational Journal of Computer Vision, 2014
- Frontiers in Neuromorphic EngineeringFrontiers in Neuroscience, 2011