ASAnand Subramoney

Anand Subramoney

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Anand Subramoney has a total of 20 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Probing Length Generalization in Mamba via Image Reconstruction

    by Jan Rathjens, Robin Schiewer, Laurenz Wiskott, Anand Subramoney

  • Open Access

    Beyond Weights: Deep learning in Spiking Neural Networks with pure synaptic-delay training

    by Edoardo W. Grappolini, Anand Subramoney

    Biological evidence suggests that adaptation of synaptic delays on short to medium timescales plays an important role in learning in the brain. Inspired by biology, we explore the feasibility and power of using synaptic delays to solve challenging tasks even when the synaptic weights are not trained but kept at randomly chosen fixed values. We show that training ONLY the delays in feed-forward spiking networks using backpropagation can achieve performance comparable to the more conventional weight training. Moreover, further constraining the weights to ternary values does not significantly affect the networks' ability to solve the tasks using only the synaptic delays. We demonstrate the task performance of delay-only training on MNIST and Fashion-MNIST datasets in preliminary experiments. This demonstrates a new paradigm for training spiking neural networks and sets the stage for models that can be more efficient than the ones that use weights for computation.

  • Open Access

    Efficient Real Time Recurrent Learning through combined activity and parameter sparsity

    by Anand Subramoney

    Backpropagation through time (BPTT) is the standard algorithm for training recurrent neural networks (RNNs), which requires separate simulation phases for the forward and backward passes for inference and learning, respectively. Moreover, BPTT requires storing the complete history of network states between phases, with memory consumption growing proportional to the input sequence length. This makes BPTT unsuited for online learning and presents a challenge for implementation on low-resource real-time systems. Real-Time Recurrent Learning (RTRL) allows online learning, and the growth of required memory is independent of sequence length. However, RTRL suffers from exceptionally high computational costs that grow proportional to the fourth power of the state size, making RTRL computationally intractable for all but the smallest of networks. In this work, we show that recurrent networks exhibiting high activity sparsity can reduce the computational cost of RTRL. Moreover, combining activity and parameter sparsity can lead to significant enough savings in computational and memory costs to make RTRL practical. Unlike previous work, this improvement in the efficiency of RTRL can be achieved without using any approximations for the learning process.

  • Open Access

    Spike frequency adaptation supports network computations on temporally dispersed information

    by Darjan Salaj, Anand Subramoney, Ceca Kraisnikovic, Guillaume Bellec, Robert Legenstein, Wolfgang Maass

    For solving tasks such as recognizing a song, answering a question, or inverting a sequence of symbols, cortical microcircuits need to integrate and manipulate information that was dispersed over time during the preceding seconds. Creating biologically realistic models for the underlying computations, especially with spiking neurons and for behaviorally relevant integration time spans, is notoriously difficult. We examine the role of spike frequency adaptation in such computations and find that it has a surprisingly large impact. The inclusion of this well-known property of a substantial fraction of neurons in the neocortex – especially in higher areas of the human neocortex – moves the performance of spiking neural network models for computations on network inputs that are temporally dispersed from a fairly low level up to the performance level of the human brain.


  • Open Access

    Self-supervised learning of probabilistic prediction through synaptic plasticity in apical dendrites: A normative model

    by Arjun Rao, Robert Legenstein, Anand Subramoney, Wolfgang Maass

    Sensory information is processed by the brain not in a simple feedforward fashion. Rather, bottom-up inputs are combined in pyramidal cells of sensory cortices with top-down information from higher brain areas that arrives through synapses in apical dendrites. The exact functional role of these top-down inputs has remained unknown. A promising abstract model posits that they provide probabilistic priors for bottom-up sensory inputs. We show that this hypothesis is consistent with a large number of experimental about synaptic plasticity in apical dendrites, in particular with the prominent role of NMDA-spikes. We identify conditions under which this synaptic plasticity could approximate the gold standard for self-supervised learning of probabilistic priors: logistic regression. Furthermore, this perspective suggests an additional functional role for the complex structure of the dendritic arborization plays: It enables the neuron to learn substantially more complex landscapes of probabilistic priors.


  • Open Access

    Revisiting the role of synaptic plasticity and network dynamics for fast learning in spiking neural networks

    by Anand Subramoney, Guillaume Bellec, Franz Scherr, Robert Legenstein, Wolfgang Maass

    Spike-based neural network models have so far not been able to reproduce the capability of the brain to learn from very few, often even from just a single example. We show that this deficiency of models disappears if one allows synaptic weights to store priors and other information that optimize the learning process, while using the network state to quickly absorb information from new examples. For that, it suffices to include biologically realistic neurons with spike frequency adaptation in the neural network model, and to optimize the learning process through meta-learning. We demonstrate this on a variety of tasks, including fast learning and deletion of attractors, adaptation of motor control to changes in the body, and solving the Morris water maze task – a paradigm for fast learning of navigation to a new goal. Significance Statement It has often been conjectured that STDP or other rules for synaptic plasticity can only explain some of the learning capabilities of brains. In particular, learning a new task from few trials is likely to engage additional mechanisms. Results from machine learning show that artificial neural networks can learn from few trials by storing information about them in their network state, rather than encoding them in synaptic weights. But these machine learning methods require neural networks with biologically unrealistic LSTM (Long Short Term Memory) units. We show that biologically quite realistic models for neural networks of the brain can exhibit similar capabilities. In particular, these networks are able to store priors that enable learning from very few examples.


  • Top Cited Papers
    Open Access

    A solution to the learning dilemma for recurrent networks of spiking neurons

    by Guillaume Bellec, Franz Scherr, Anand Subramoney, Elias Hajek, Darjan Salaj, Robert Legenstein, Wolfgang Maass

    Recurrently connected networks of spiking neurons underlie the astounding information processing capabilities of the brain. Yet in spite of extensive research, how they can learn through synaptic plasticity to carry out complex network computations remains unclear. We argue that two pieces of this puzzle were provided by experimental data from neuroscience. A mathematical result tells us how these pieces need to be combined to enable biologically plausible online network learning through gradient descent, in particular deep reinforcement learning. This learning method–called e-prop–approaches the performance of backpropagation through time (BPTT), the best-known method for training recurrent neural networks in machine learning. In addition, it suggests a method for powerful on-chip learning in energy-efficient spike-based hardware for artificial intelligence.


  • Open Access

    Spike frequency adaptation supports network computations on temporally dispersed information

    by Darjan Salaj, Anand Subramoney, Ceca Kraišniković, Guillaume Bellec, Robert Legenstein, Wolfgang Maass

    For solving tasks such as recognizing a song, answering a question, or inverting a sequence of symbols, cortical microcircuits need to integrate and manipulate information that was dispersed over time during the preceding seconds. Creating biologically realistic models for the underlying computations, especially with spiking neurons and for behaviorally relevant integration time spans, is notoriously difficult. We examine the role of spike frequency adaptation in such computations and find that it has a surprisingly large impact. The inclusion of this well known property of a substantial fraction of neurons in the neocortex — especially in higher areas of the human neocortex — moves the performance of spiking neural network models for computations on network inputs that are temporally dispersed from a fairly low level up to the performance level of the human brain.