Abstract
Spiking Neural Networks (SNNs) have garnered significant attention due to their biological plausibility and energy efficiency. However, their generative modeling capabilities remain relatively underexplored, particularly in the context of handling temporal information. In this work, we propose a novel SNN-based Vector Quantized-Variational Autoencoder (VQ-VAE) framework that incorporates a temporal codebook inspired by hippocampal time cells. The temporal codebook enables the model to effectively encode and utilize temporal dependencies, facilitating the generation of high-resolution, temporally consistent images and event-based data. Extensive experiments conducted on diverse datasets, including CelebA-HQ, LSUN Bedroom, LSUN Church, CIFAR-10, and N-MNIST, demonstrate the superior performance of the proposed framework. Our method achieves state-of-the-art results in image quality and temporal consistency, outperforming existing SNN-based generative models. Furthermore, through destructive experiments, we validate the critical role of the temporal codebook in enhancing temporal correlation and improving generative performance. In addition to its superior image generation capabilities, our model exhibits significantly lower energy consumption compared to ANN-based counterparts, leveraging the inherent sparsity of SNNs. These findings highlight the potential of integrating temporal dynamics into the latent space of generative models, paving the way for advancements in neuromorphic computing and biologically inspired machine learning.
Funding Information
  • National Natural Science Foundation of China (62406325)

This publication has 39 references indexed in Scilit: