RMP-Loss: Regularizing Membrane Potential Distribution for Spiking Neural Networks
Authors:
- 1 October 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 17345-17355
Abstract
Spiking Neural Networks (SNNs) as one of the biology-inspired models have received much attention recently. It can significantly reduce energy consumption since they quantize the real-valued membrane potentials to 0/1 spikes to transmit information thus the multiplications of activations and weights can be replaced by additions when implemented on hardware. However, this quantization mechanism will inevitably introduce quantization error, thus causing catastrophic information loss. To address the quantization error problem, we propose a regularizing membrane potential loss (RMP-Loss) to adjust the distribution which is directly related to quantization error to a range close to the spikes. Our method is extremely simple to implement and straightforward to train an SNN. Furthermore, it is shown to consistently outperform previous state-of-the-art methods over different network architectures and datasets.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China
This publication has 41 references indexed in Scilit:
- Spiking neural predictive coding for continually learning from data streamsNeurocomputing, 2023
- Joint A-SNN: Joint training of artificial and spiking neural networks via self-Distillation and weight factorizationPattern Recognition, 2023
- Reducing Information Loss for Spiking Neural Networks2022
- Real Spike: Learning Real-Valued Spikes for Spiking Neural Networks2022
- Navigating Local Minima in Quantized Spiking Neural Networks2022
- Deep Spiking Neural Network: Energy Efficiency Through Time Based Coding2020
- Towards artificial general intelligence with hybrid Tianjic chip architectureNature, 2019
- AutoAugment: Learning Augmentation Strategies From Data2019
- Error-Backpropagation in Networks of Fractionally Predictive Spiking Neurons2011
- ImageNet: A large-scale hierarchical image database2009