Davide Liberato Manna

Timely and energy-efficient time series forecasting can play a key role on edge devices, where power requirements can be stringent. Spiking Neural Networks (SNNs) are regarded as a new avenue in which to solve time series problems, but with lower SWaP (Size, Weight, and Power) needs. We propose an SNN pipeline to process and forecast time series, developing a novel data spike-encoding mechanism and two loss functions that optimise the prediction of the upcoming spikes. Our approach encodes a signal into sequences of spikes that approximate its derivative, preparing the data to be processed by the SNN, while our proposed loss functions account for the reconstruction of the output spikes into a meaningful value to promote convergence to top-level solutions. Results show that our solution can effectively learn from the encoded data and the SNN trained with our loss function can outperform the same model trained with SLAYER's default loss.

The potential of neuromorphic (NM) solutions often lies in their low-SWaP (Size, Weight, and Power) capabilities, which often drive their application to domains that could benefit from this. Nevertheless, spiking neural networks (SNNs), with their inherent time-based nature, present an attractive alternative also for areas where data features are present in the time dimension, such as time series forecasting. Time series data, characterized by seasonality and trends, can benefit from the unique processing capabilities of SNNs, which offer a novel approach for this type of task. Additionally, time series data can serve as a benchmark for evaluating SNN performance, providing a valuable alternative to traditional datasets. However, the challenge lies in the real-valued nature of time series data, which is not inherently suited for SNN processing. In this work, we propose a novel spike-encoding mechanism and two loss functions to address this challenge. Our encoding system, inspired by NM event-based sensors, converts the derivative of a signal into spikes, enhancing interoperability with the NM technology and also making the data suitable for SNN processing. Our loss functions then optimize the learning of subsequent spikes by the SNN. We train a simple SNN using SLAYER as a learning rule and conduct experiments using two electricity load forecasting datasets. Our results demonstrate that SNNs can effectively learn from encoded data, and our proposed DecodingLoss function consistently outperforms SLAYER’s SpikeTime loss function. This underscores the potential of SNNs for time series forecasting and sets the stage for further research in this promising area of research.

Size, Weight, and Power (SWaP) concerns are growing as artificial intelligence (AI) use spreads in edge applications. AI algorithms, such as artificial neural networks (ANNs), have revolutionized many fields, e.g. computer vision (CV), but at a large computational/power burden. Biological intelligence is notably more computationally efficient. Neuromorphic edge processors and spiking neural networks (SNNs) aim to follow biology closer with spike-based operations resulting in sparsity and lower-SWaP operations than traditional ANNs with SNNs only "firing/spiking" when needed. Understanding the trade space of SWaP when embracing neuromorphic computing has not been studied heavily. To addresses this, we present a repeatable and scalable apples-to-apples comparison of traditional ANNs and SNNs for edge processing with demonstration on both classical and neuromorphic edge hardware. Results show that SNNs combined with neuromorphic hardware can provide comparable accuracy for CV to ANNs at 1/10 th the power.

Size, Weight, and Power (SWaP) concerns are growing as artificial intelligence (AI) use spreads in edge applications. AI algorithms, such as artificial neural networks (ANNs), have revolutionized many fields, e.g. computer vision (CV), but at a large computational/power burden. Biological intelligence is notably more computationally efficient. Neuromorphic edge processors and spiking neural networks (SNNs) aim to follow biology closer with spike-based operations resulting in sparsity and lower-SWaP operations than traditional ANNs with SNNs only "firing/spiking" when needed. Understanding the trade space of SWaP when embracing neuromorphic computing has not been studied heavily. To addresses this, we present a repeatable and scalable apples-to-apples comparison of traditional ANNs and SNNs for edge processing with demonstration on both classical and neuromorphic edge hardware. Results show that SNNs combined with neuromorphic hardware can provide comparable accuracy for CV to ANNs at 1/10 th the power.

Developing effective learning systems for Machine Learning (ML) applications in the Neuromorphic (NM) field requires extensive experimentation and simulation. Software frameworks aid and ease this process by providing a set of ready-to-use tools that researchers can leverage. The recent interest in NM technology has seen the development of several new frameworks that do this, and that add up to the panorama of already existing libraries that belong to neuroscience fields. This work reviews 9 frameworks for the development of Spiking Neural Networks (SNNs) that are specifically oriented towards data science applications. We emphasize the availability of spiking neuron models and learning rules to more easily direct decisions on the most suitable frameworks to carry out different types of research. Furthermore, we present an extension to the SpykeTorch framework that gives users access to a much broader choice of neuron models to embed in SNNs and make the code publicly available.

Spiking neural networks (SNNs) are largely inspired by biology and neuroscience and leverage ideas and theories to create fast and efficient learning systems. Spiking neuron models are adopted as core processing units in neuromorphic systems because they enable event-based processing. Among many neuron models, the integrate-and-fire (I&F) models are often adopted, with the simple leaky I&F (LIF) being the most used. The reason for adopting such models is their efficiency and/or biological plausibility. Nevertheless, rigorous justification for adopting LIF over other neuron models for use in artificial learning systems has not yet been studied. This work considers various neuron models in the literature and then selects computational neuron models that are single-variable, efficient, and display different types of complexities. From this selection, we make a comparative study of three simple I&F neuron models, namely the LIF, the quadratic I&F (QIF) and the exponential I&F (EIF), to understand whether the use of more complex models increases the performance of the system and whether the choice of a neuron model can be directed by the task to be completed. Neuron models are tested within an SNN trained with spike-timing dependent plasticity (STDP) on a classification task on the N-MNIST and DVS gestures datasets. Experimental results reveal that more complex neurons manifest the same ability as simpler ones to achieve high levels of accuracy on a simple dataset (N-MNIST), albeit requiring comparably more hyper-parameter tuning. However, when the data possess richer spatio-temporal features, the QIF and EIF neuron models steadily achieve better results. This suggests that accurately selecting the model based on the richness of the feature spectrum of the data could improve the whole system’s performance. Finally, the code implementing the spiking neurons in the SpykeTorch framework is made publicly available.

Spiking neural networks (SNNs) have become an interesting alternative to conventional artificial neural networks (ANN) thanks to their temporal processing capabilities and energy efficient implementations in neuromorphic hardware. However, the challenges involved in training SNNs have limited their performance in terms of accuracy and thus their applications. Improving learning algorithms and neural architectures for a more accurate feature extraction is therefore one of the current priorities in SNN research. In this paper we present a study on the key components of modern spiking architectures. We design a spiking version of the successful residual network architecture and provide an in-depth study on the possible implementations of spiking residual connections. This study shows how, depending on the use case, the optimal residual connection implementation may vary. Additionally, we empirically compare different techniques in image classification datasets taken from the best performing networks. Our results provide a state of the art guide to SNN design, which allows to make informed choices when trying to build the optimal visual feature extractor. Finally, our network outperforms previous SNN architectures in CIFAR-10 (94.14%) and CIFAR-100 (74.65%) datasets and matches the state of the art in DVS-CIFAR10 (72.98%), with less parameters than the previous state of the art and without the need for ANN–SNN conversion. Code available at: https://github.com/VicenteAlex/Spiking_ResNet.

Spiking neural networks (SNNs) are largely inspired by biology and neuroscience and leverage ideas and theories to create fast and efficient learning systems. Spiking neuron models are adopted as core processing units in neuromorphic systems because they enable event-based processing. The integrate-and-fire (I&F) models are often adopted, with the simple Leaky I&F (LIF) being the most used. The reason for adopting such models is their efficiency and/or biological plausibility. Nevertheless, rigorous justification for adopting LIF over other neuron models for use in artificial learning systems has not yet been studied. This work considers various neuron models in the literature and then selects computational neuron models that are single-variable, efficient, and display different types of complexities. From this selection, we make a comparative study of three simple I&F neuron models, namely the LIF, the Quadratic I&F (QIF) and the Exponential I&F (EIF), to understand whether the use of more complex models increases the performance of the system and whether the choice of a neuron model can be directed by the task to be completed. Neuron models are tested within an SNN trained with Spike-Timing Dependent Plasticity (STDP) on a classification task on the N-MNIST and DVS Gestures datasets. Experimental results reveal that more complex neurons manifest the same ability as simpler ones to achieve high levels of accuracy on a simple dataset (N-MNIST), albeit requiring comparably more hyper-parameter tuning. However, when the data possess richer Spatio-temporal features, the QIF and EIF neuron models steadily achieve better results. This suggests that accurately selecting the model based on the richness of the feature spectrum of the data could improve the whole system's performance. Finally, the code implementing the spiking neurons in the SpykeTorch framework is made publicly available.