A Biologically Inspired Attention Model for Neural Signal Analysis
Preprint
Abstract
Understanding how the brain represents sensory information and triggers behavioural responses is a fundamental goal in neuroscience. Recent advances in neuronal recording techniques aim to progress towards this milestone, yet the resulting high-dimensional responses are challenging to interpret and link to relevant variables. Although existing machine learning models propose to do so, they often sacrifice interpretability for predictive power, effectively operating as black boxes. In this work, we introduce SPARKS, a biologically inspired model capable of high decoding accuracy and interpretable discovery within a single framework. SPARKS adapts the self-attention mechanism of large language models to extract information from the timing of single spikes and the sequence in which neurons fire using Hebbian learning. Trained with a criterion inspired by predictive coding to enforce temporal coherence, our model produces low-dimensional latent embeddings that are robust across sessions and animals. By directly capturing the underlying data distribution through a generative encoding-decoding framework, SPARKS matches or exceeds state-of-the-art decoders across diverse electrophysiology and calcium imaging datasets from the motor, visual and entorhinal cortices. Crucially, the Hebbian coefficients learned by the model are interpretable, allowing us to infer directed functional interactions and recover the known functional hierarchy of the mouse visual cortex. Overall, SPARKS unifies representation learning, high-performance decoding and model interpretability in a single framework by bridging neuroscience and AI, providing a powerful and versatile tool for dissecting neural computations and marking a step towards the next generation of biologically inspired intelligent systems.Keywords
Funding Information
- BBSRC (BB/X013472/1)
- Wellcome Trust
- The Francis Crick Institute (CC2118)
This publication has 69 references indexed in Scilit:
- Diesel2p mesoscope with dual independent scan engines for flexible capture of dynamics in distributed neural circuitryNature Communications, 2021
- Representational drift in the mouse visual cortexCurrent Biology, 2021
- Stimulus-dependent representational drift in primary visual cortexNature Communications, 2021
- Hierarchical organization of cortical and thalamic connectivityNature, 2019
- Cortical population activity within a preserved neural manifold underlies multiple motor behaviorsNature Communications, 2018
- Dynamic Reorganization of Neuronal Activity Patterns in Parietal CortexCell, 2017
- Steady or changing? Long-term monitoring of neuronal population activityTrends in Neurosciences, 2013
- Memory, navigation and theta rhythm in the hippocampal-entorhinal systemNature Neuroscience, 2013
- Neural population dynamics during reachingNature, 2012
- Time course of visual perception: Coarse-to-fine processing and beyondProgress in Neurobiology, 2008