TSTerrence J. Sejnowski

Terrence J. Sejnowski

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Terrence J. Sejnowski has a total of 14 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • The Memory Paradox: Why Our Brains Need Knowledge in an Age of AI

    by Barbara Oakley, Michael Johnston, Ken-Zen Chen, Eulho Jung, Terrence Sejnowski


  • Optimizing Attention and Cognitive Control Costs Using Temporally Layered Architectures

    by Devdhar Patel, Terrence Sejnowski, Hava Siegelmann

    The current reinforcement learning framework focuses exclusively on performance, often at the expense of efficiency. In contrast, biological control achieves remarkable performance while also optimizing computational energy expenditure and decision frequency. We propose a decision-bounded Markov decision process (DB-MDP) that constrains the number of decisions and computational energy available to agents in reinforcement learning environments. Our experiments demonstrate that existing reinforcement learning algorithms struggle within this framework, leading to either failure or suboptimal performance. To address this, we introduce a biologically inspired, temporally layered architecture (TLA), enabling agents to manage computational costs through two layers with distinct timescales and energy requirements. TLA achieves optimal performance in decision-bounded environments and in continuous control environments, matching state-of-the-art performance while using a fraction of the computing cost. Compared to current reinforcement learning algorithms that solely prioritize performance, our approach significantly lowers computational energy expenditure while maintaining performance. These findings establish a benchmark and pave the way for future research on energy and time-aware control.


  • Open Access

    Hidden Traveling Waves bind Working Memory Variables in Recurrent Neural Networks

    by Arjun Karuvally, Terrence J. Sejnowski, Hava T. Siegelmann

    Traveling waves are a fundamental phenomenon in the brain, playing a crucial role in short-term information storage. In this study, we leverage the concept of traveling wave dynamics within a neural lattice to formulate a theoretical model of neural working memory, study its properties, and its real world implications in AI. The proposed model diverges from traditional approaches, which assume information storage in static, register-like locations updated by interference. Instead, the model stores data as waves that is updated by the wave's boundary conditions. We rigorously examine the model's capabilities in representing and learning state histories, which are vital for learning history-dependent dynamical systems. The findings reveal that the model reliably stores external information and enhances the learning process by addressing the diminishing gradient problem. To understand the model's real-world applicability, we explore two cases: linear boundary condition (LBC) and non-linear, self-attention-driven boundary condition (SBC). The model with the linear boundary condition results in a shift matrix plus low-rank matrix currently used in H3 state space RNN. Further, our experiments with LBC reveal that this matrix is effectively learned by Recurrent Neural Networks (RNNs) through backpropagation when modeling history-dependent dynamical systems. Conversely, the SBC parallels the autoregressive loop of an attention-only transformer with the context vector representing the wave substrate. Collectively, our findings suggest the broader relevance of traveling waves in AI and its potential in advancing neural network architectures.

  • Open Access

    Energy-based General Sequential Episodic Memory Networks at the Adiabatic Limit

    by Arjun Karuvally, Terry J. Sejnowski, Hava T. Siegelmann

    The General Associative Memory Model (GAMM) has a constant state-dependant energy surface that leads the output dynamics to fixed points, retrieving single memories from a collection of memories that can be asynchronously preloaded. We introduce a new class of General Sequential Episodic Memory Models (GSEMM) that, in the adiabatic limit, exhibit temporally changing energy surface, leading to a series of meta-stable states that are sequential episodic memories. The dynamic energy surface is enabled by newly introduced asymmetric synapses with signal propagation delays in the network's hidden layer. We study the theoretical and empirical properties of two memory models from the GSEMM class, differing in their activation functions. LISEM has non-linearities in the feature layer, whereas DSEM has non-linearity in the hidden layer. In principle, DSEM has a storage capacity that grows exponentially with the number of neurons in the network. We introduce a learning rule for the synapses based on the energy minimization principle and show it can learn single memories and their sequential relationships online. This rule is similar to the Hebbian learning algorithm and Spike-Timing Dependent Plasticity (STDP), which describe conditions under which synapses between neurons change strength. Thus, GSEMM combines the static and dynamic properties of episodic memory under a single theoretical framework and bridges neuroscience, machine learning, and artificial intelligence.

  • Open Access

    Laws for Glia Organization Conserved Across Mammals

    by Antonio Pinto-Duarte, Katharine Bogue, Terrence J. Sejnowski, Shyam Srinivasan

    The organizational principles of glia remain largely unknown despite their vital role in nervous system function. Previous work has shown that the number of glia per unit volume of neocortex is constant across mammalian species. We hypothesize that the conservation of glia volume density within brain regions might be a governing principle of organization across species. To test this hypothesis, we used stereology, light microscopy, and data available in the literature to examine five brain regions: the cerebral cortex and four brain regions that differ from the cerebral cortex and each other - the anterior piriform cortex, the posterior piriform cortex, the entorhinal cortex, and the cerebellum. We discovered two orderly relationships: First, glia volume density within a brain region was constant across species, including humans, although it significantly differed between regions, suggesting that glia density might constitute a region-specific marker. Second, the ratio of glia to neuron increased with brain volume according to a ¼ power law in the primate frontal cortex and the neocortex, the mammalian paleocortex, and the cerebellum. These relationships show that the development of glia and neurons are coupled, and suggest that what a neural circuit computes depends as much on its glial components as on its neurons. Main Points: The volume density of glia (i.e., number of glia per unit volume) within a brain region is con-served across mammalian species including humans. The ratio of glia to neuron increases with bigger brains. The volume density of glia is significantly different across functionally and architecturally dif-ferent brain regions and could function as a region-specific marker. Glia obey scaling constraints that are different from scaling constraints for neurons.

  • Efficient models of polymerization applied to FtsZ ring assembly in Escherichia coli

    by Álvaro Ruiz-Martínez, Thomas M. Bartol, Terrence J. Sejnowski, Daniel M. Tartakovsky

    High protein concentrations complicate modeling of polymer assembly kinetics by introducing structural complexity and a large variety of protein forms. We present a modeling approach that achieves orders of magnitude speed-up by replacing distributions of lengths and widths with their average counterparts and by introducing a hierarchical classification of species and reactions into sets. We have used this model to study FtsZ ring assembly in Escherichia coli. The model’s prediction of key features of the ring formation, such as time to reach the steady state, total concentration of FtsZ species in the ring, total concentration of monomers, and average dimensions of filaments and bundles, are all in agreement with the experimentally observed values. Besides validating our model against the in vivo observations, this study fills some knowledge gaps by proposing a specific structure of the ring, describing the influence of the total concentration in short and long kinetics processes, determining some characteristic mechanisms in polymer assembly regulation, and providing insights about the role of ZapA proteins, critical components for both positioning and stability of the ring. Significance Our modeling framework yields accurate and computationally efficient quantitative predictions of complex kinetics of polymerization processes in biological systems. The resulting model consists of 10 differential equations, regardless of the total concentration of proteins. This is in contrast to previous polymerization models, in which the number of equations increases with the total concentrations, reaching into the thousands. Consequently, our model is orders of magnitude faster than its existing alternatives. It can be used to predict polymerization kinetics at high concentrations characteristic of in vivo processes and, especially, their compartmentalized and spatially distributed representations.