Sebastian Siegel
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Sebastian Siegel has a total of 40 co-authors. Here are the top co-authors by number of shared publications:
- Dirk J. Wouters
Institute of Electronic Materials (IWE 2) and JARA-FIT, RWTH Aachen University Aachen, Aachen, Germany
5 shared publications - Rainer Waser
Peter-Grnberg-Institut 7 (PGI-7), Forschungszentrum Jlich GmbH, Jlich, Germany
5 shared publications - Regina Dittmann
Forschungszentrum Jülich
4 shared publications - John Paul Strachan
Peter Grünberg Institute (PGI-14), Forschungszentrum Jülich GmbH, Jülich, Germany
3 shared publications - Stephan Menzel
Peter Grünberg Institut (PGI-7), Forschungszentrum Jülich, Jülich, Germany
3 shared publications - Tobias Ziegler
JARA-FIT and Institute of Materials in Electrical Engineering and Information Technology II RWTH Aachen University Sommerfeldstraße 24 Aachen 52074 Germany
3 shared publications
Latest Publications
- Open Access
- 1 March 2026
- Published by Wiley in Advanced Intelligent Systems
- Vol. 8 (5)
A Flexible and Energy‐Efficient Compute‐in‐Memory Accelerator for Kolmogorov–Arnold Networks
Emerging Kolmogorov–Arnold networks (KANs) replace the linear weights of neural networks with trainable nonlinear functions. This modification is particularly attractive for scientific computing, where KANs can match the accuracy of conventional multilayer perceptrons (MLPs) while reducing model size by up to 100×. However, this efficiency comes at the cost of computationally expensive nonlinear evaluations, unlike conventional MLPs dominated by linear matrix multiplications. We present a flexible and energy‐efficient compute‐in‐memory accelerator tailored for KANs, developed through cross‐layer optimization across algorithm, architecture, circuit, and device levels. The accelerator computes arbitrary nonlinear functions using a single‐read scheme and read‐optimized memory arrays with nonvolatile memristive devices. Our system achieves a lowest energy of 8.69 pJ per KAN function. In terms of energy‐delay product, it provides 1996× improvement over CPUs, 208× over standard MLP‐oriented compute‐in‐memory accelerators, and up to 71× over prior KAN accelerators. These results establish energy‐efficient hardware primitives for implementing advanced nonlinear networks in scientific computing.
- 1 March 2026
- Published by Wiley in Advanced Intelligent Systems
- Vol. 8 (5)
Open AccessA Flexible and Energy‐Efficient Compute‐in‐Memory Accelerator for Kolmogorov–Arnold Networks
Emerging Kolmogorov–Arnold networks (KANs) replace the linear weights of neural networks with trainable nonlinear functions. This modification is particularly attractive for scientific computing, where KANs can match the accuracy of conventional multilayer perceptrons (MLPs) while reducing model size by up to 100×. However, this efficiency comes at the cost of computationally expensive nonlinear evaluations, unlike conventional MLPs dominated by linear matrix multiplications. We present a flexible and energy‐efficient compute‐in‐memory accelerator tailored for KANs, developed through cross‐layer optimization across algorithm, architecture, circuit, and device levels. The accelerator computes arbitrary nonlinear functions using a single‐read scheme and read‐optimized memory arrays with nonvolatile memristive devices. Our system achieves a lowest energy of 8.69 pJ per KAN function. In terms of energy‐delay product, it provides 1996× improvement over CPUs, 208× over standard MLP‐oriented compute‐in‐memory accelerators, and up to 71× over prior KAN accelerators. These results establish energy‐efficient hardware primitives for implementing advanced nonlinear networks in scientific computing.
- Open Access
- 6 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
- Vol. 16 (2) , 483-499
Algorithm-Hardware Implications of Softmax Approximations for In-Memory Computing-Based LLM Accelerators
As large language models (LLMs) continue to scale in both size and particularly in sequence length, the softmax-attention mechanism of the underlying transformer architecture has emerged as a critical bottleneck. While in-memory computing (IMC) based architectures offer efficient acceleration for vector-matrix multiplications (VMMs), integrating softmax remains challenging due to its computational complexity. This work presents a comprehensive analysis of softmax approximations and softmax precision requirements in modern LLMs spanning model sizes from 125M to 405B parameters across several natural language processing (NLP) benchmarks. Based on these findings, we present an IMC-optimized reformulation of softmax for attention that results in negligible accuracy degradation, including (a) transformation of complexity from sequence-length-dependency to the much smaller attention head dimension and (b) merging of softmax components within the adjacent IMC’s VMM operations. We present both digital and analog implementations. Our digital approach reduces area overhead to 81.2 μm2 and energy to 0.69 pJ per input-output pair, resulting in more than 20× lower energy-delay-area product compared to state-of-the-art. Our analog implementation achieves 37.1 μm2 and 0.33 pJ by leveraging a spike-based encoding. These results highlight the importance of rethinking softmax computation for IMC to enable scalable transformer accelerators.
- 6 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
- Vol. 16 (2) , 483-499
Open AccessAlgorithm-Hardware Implications of Softmax Approximations for In-Memory Computing-Based LLM Accelerators
As large language models (LLMs) continue to scale in both size and particularly in sequence length, the softmax-attention mechanism of the underlying transformer architecture has emerged as a critical bottleneck. While in-memory computing (IMC) based architectures offer efficient acceleration for vector-matrix multiplications (VMMs), integrating softmax remains challenging due to its computational complexity. This work presents a comprehensive analysis of softmax approximations and softmax precision requirements in modern LLMs spanning model sizes from 125M to 405B parameters across several natural language processing (NLP) benchmarks. Based on these findings, we present an IMC-optimized reformulation of softmax for attention that results in negligible accuracy degradation, including (a) transformation of complexity from sequence-length-dependency to the much smaller attention head dimension and (b) merging of softmax components within the adjacent IMC’s VMM operations. We present both digital and analog implementations. Our digital approach reduces area overhead to 81.2 μm2 and energy to 0.69 pJ per input-output pair, resulting in more than 20× lower energy-delay-area product compared to state-of-the-art. Our analog implementation achieves 37.1 μm2 and 0.33 pJ by leveraging a spike-based encoding. These results highlight the importance of rethinking softmax computation for IMC to enable scalable transformer accelerators.
- Open Access
- 27 January 2026
- Published by Wiley in Advanced Intelligent Systems
- Vol. 8 (4)
QS4D: Quantization‐Aware Training for Efficient Hardware Deployment of Structured State‐Space Sequential Models
Structured state space models (SSM) have recently emerged as a new class of deep learning models, particularly well‐suited for processing long sequences. Their constant memory footprint, in contrast to the linearly scaling memory demands of transformers, makes them attractive candidates for deployment on resource‐constrained edge‐computing devices. While recent works have explored the effect of quantization‐aware training (QAT) on SSMs, they typically do not address its implications for specialized edge hardware, for example, analog in‐memory computing (AIMC) chips. In this work, it is demonstrated that QAT can significantly reduce the complexity of SSMs by up to two orders of magnitude across various performance metrics. The relation between model size and numerical precision is analyzed, and it is shown that QAT enhances robustness to analog noise and enables structural pruning. Finally, these techniques are integrated to deploy SSMs on a memristive AIMC substrate and highlight the resulting benefits in terms of computational efficiency.
- 27 January 2026
- Published by Wiley in Advanced Intelligent Systems
- Vol. 8 (4)
Open AccessQS4D: Quantization‐Aware Training for Efficient Hardware Deployment of Structured State‐Space Sequential Models
Structured state space models (SSM) have recently emerged as a new class of deep learning models, particularly well‐suited for processing long sequences. Their constant memory footprint, in contrast to the linearly scaling memory demands of transformers, makes them attractive candidates for deployment on resource‐constrained edge‐computing devices. While recent works have explored the effect of quantization‐aware training (QAT) on SSMs, they typically do not address its implications for specialized edge hardware, for example, analog in‐memory computing (AIMC) chips. In this work, it is demonstrated that QAT can significantly reduce the complexity of SSMs by up to two orders of magnitude across various performance metrics. The relation between model size and numerical precision is analyzed, and it is shown that QAT enhances robustness to analog noise and enables structural pruning. Finally, these techniques are integrated to deploy SSMs on a memristive AIMC substrate and highlight the resulting benefits in terms of computational efficiency.
- Open Access
IMSSA: Deploying modern state-space models on memristive in-memory compute hardware
Processing long temporal sequences is a key challenge in deep learning. In recent years, Transformers have become state-of-the-art for this task, but suffer from excessive memory requirements due to the need to explicitly store the sequences. To address this issue, structured state-space sequential (S4) models recently emerged, offering a fixed memory state while still enabling the processing of very long sequence contexts. The recurrent linear update of the state in these models makes them highly efficient on modern graphics processing units (GPU) by unrolling the recurrence into a convolution. However, this approach demands significant memory and massively parallel computation, which is only available on the latest GPUs. In this work, we aim to bring the power of S4 models to edge hardware by significantly reducing the size and computational demand of an S4D model through quantization-aware training, even achieving ternary weights for a simple real-world task. To this end, we extend conventional quantization-aware training to tailor it for analog in-memory compute hardware. We then demonstrate the deployment of recurrent S4D kernels on memrisitve crossbar arrays, enabling their computation in an in-memory compute fashion. To our knowledge, this is the first implementation of S4 kernels on in-memory compute hardware.
Open AccessIMSSA: Deploying modern state-space models on memristive in-memory compute hardware
Processing long temporal sequences is a key challenge in deep learning. In recent years, Transformers have become state-of-the-art for this task, but suffer from excessive memory requirements due to the need to explicitly store the sequences. To address this issue, structured state-space sequential (S4) models recently emerged, offering a fixed memory state while still enabling the processing of very long sequence contexts. The recurrent linear update of the state in these models makes them highly efficient on modern graphics processing units (GPU) by unrolling the recurrence into a convolution. However, this approach demands significant memory and massively parallel computation, which is only available on the latest GPUs. In this work, we aim to bring the power of S4 models to edge hardware by significantly reducing the size and computational demand of an S4D model through quantization-aware training, even achieving ternary weights for a simple real-world task. To this end, we extend conventional quantization-aware training to tailor it for analog in-memory compute hardware. We then demonstrate the deployment of recurrent S4D kernels on memrisitve crossbar arrays, enabling their computation in an in-memory compute fashion. To our knowledge, this is the first implementation of S4 kernels on in-memory compute hardware.
- Open Access
- 19 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Electron Devices
- Vol. 71 (9) , 5345-5353
Synaptogen: A Cross-Domain Generative Device Model for Large-Scale Neuromorphic Circuit Design
We present a fast generative modeling approach for resistive memories that reproduces the complex statistical properties of real-world devices. By training on extensive measurement data of an integrated 1T1R array (6000 cycles of 512 devices), an autoregressive stochastic process accurately accounts for the cross-correlations between device switching parameters, while nonlinear transformations ensure agreement with the joint cycle-to-cycle (C2C) and device-to-device (D2D) write distributions. In addition to a high-level programming version, the model is also implemented in Verilog-A to enable efficient simulation of analog circuits. This statistically comprehensive model can be used to simulate crossbar sizes with up to $1024\times 1024$ devices, and benchmarks show that it achieves read/write speeds several orders of magnitude higher than a variability-aware physics-based compact model and over $10\times $ faster than even a simplified and deterministic compact model.
- 19 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Electron Devices
- Vol. 71 (9) , 5345-5353
Open AccessSynaptogen: A Cross-Domain Generative Device Model for Large-Scale Neuromorphic Circuit Design
We present a fast generative modeling approach for resistive memories that reproduces the complex statistical properties of real-world devices. By training on extensive measurement data of an integrated 1T1R array (6000 cycles of 512 devices), an autoregressive stochastic process accurately accounts for the cross-correlations between device switching parameters, while nonlinear transformations ensure agreement with the joint cycle-to-cycle (C2C) and device-to-device (D2D) write distributions. In addition to a high-level programming version, the model is also implemented in Verilog-A to enable efficient simulation of analog circuits. This statistically comprehensive model can be used to simulate crossbar sizes with up to $1024\times 1024$ devices, and benchmarks show that it achieves read/write speeds several orders of magnitude higher than a variability-aware physics-based compact model and over $10\times $ faster than even a simplified and deterministic compact model.
- Open Access
- 17 June 2024
- Published by Wiley in Advanced Electronic Materials
- Vol. 10 (12)
Resolving the Relaxation of Volatile Valence Change Memory
Memristive devices based on the valence change mechanism are highly interesting candidates for data storage and hardware implementation of synapses in neuromorphic circuits. Although long‐term retention is often required for data storage applications, a slight resistance drift of the low resistive state (LRS) is observed even for stable devices. For other devices, the LRS has been observed to decay rapidly to the high resistive state (HRS). These types of devices are of interest for neuromorphic circuits where short‐term plasticity is required. In this work, the LRS relaxation of volatile, crystalline Pt/SrTiO3/Nb:SrTiO3: devices is investigated in detail, yielding time constants ranging from milliseconds to seconds. The decay is analyzed in terms of the Gibbs free energy gradient for the contribution of oxygen ion migration. A relaxation model based on drift‐diffusion dynamics is presented. The model may serve as a tool for developing guidelines and design rules for future volatile memristive technology based on Schottky barrier mediated electron transport.
- 17 June 2024
- Published by Wiley in Advanced Electronic Materials
- Vol. 10 (12)
Open AccessResolving the Relaxation of Volatile Valence Change Memory
Memristive devices based on the valence change mechanism are highly interesting candidates for data storage and hardware implementation of synapses in neuromorphic circuits. Although long‐term retention is often required for data storage applications, a slight resistance drift of the low resistive state (LRS) is observed even for stable devices. For other devices, the LRS has been observed to decay rapidly to the high resistive state (HRS). These types of devices are of interest for neuromorphic circuits where short‐term plasticity is required. In this work, the LRS relaxation of volatile, crystalline Pt/SrTiO3/Nb:SrTiO3: devices is investigated in detail, yielding time constants ranging from milliseconds to seconds. The decay is analyzed in terms of the Gibbs free energy gradient for the contribution of oxygen ion migration. A relaxation model based on drift‐diffusion dynamics is presented. The model may serve as a tool for developing guidelines and design rules for future volatile memristive technology based on Schottky barrier mediated electron transport.
- Open Access
- 17 September 2023
- Published by Association for Computing Machinery (ACM)
- p. 44-45
Work-in-Progress: A Universal Instrumentation Platform for Non-Volatile Memories
Emerging non-volatile memories (NVMs) represent a disruptive technology that allows a paradigm shift from the conventional von Neumann architecture towards more efficient computing-in-memory (CIM) architectures. Several instrumentation platforms have been proposed to interface NVMs allowing the characterization of single cells and crossbar structures. However, these platforms suffer from low flexibility and are not capable of performing CIM operations on NVMs. Therefore, we recently designed and built the NeuroBreakoutBoard, a highly versatile instrumentation platform capable of executing CIM on NVMs. We present our preliminary results demonstrating a relative error < 5% in the range of 1 kΩ to 1 MΩ and showcase the switching behavior of a HfO2/Ti-based memristive cell.
- 17 September 2023
- Published by Association for Computing Machinery (ACM)
- p. 44-45
Open AccessWork-in-Progress: A Universal Instrumentation Platform for Non-Volatile Memories
Emerging non-volatile memories (NVMs) represent a disruptive technology that allows a paradigm shift from the conventional von Neumann architecture towards more efficient computing-in-memory (CIM) architectures. Several instrumentation platforms have been proposed to interface NVMs allowing the characterization of single cells and crossbar structures. However, these platforms suffer from low flexibility and are not capable of performing CIM operations on NVMs. Therefore, we recently designed and built the NeuroBreakoutBoard, a highly versatile instrumentation platform capable of executing CIM on NVMs. We present our preliminary results demonstrating a relative error < 5% in the range of 1 kΩ to 1 MΩ and showcase the switching behavior of a HfO2/Ti-based memristive cell.
- Open Access
- 1 September 2023
- Published by IOP Publishing in Neuromorphic Computing and Engineering
- Vol. 3 (3) , 034014
Sequence learning in a spiking neuronal network with memristive synapses
Brain-inspired computing proposes a set of algorithmic principles that hold promise for advancing artificial intelligence. They endow systems with self learning capabilities, efficient energy usage, and high storage capacity. A core concept that lies at the heart of brain computation is sequence learning and prediction. This form of computation is essential for almost all our daily tasks such as movement generation, perception, and language. Understanding how the brain performs such a computation is not only important to advance neuroscience, but also to pave the way to new technological brain-inspired applications. A previously developed spiking neural network implementation of sequence prediction and recall learns complex, high-order sequences in an unsupervised manner by local, biologically inspired plasticity rules. An emerging type of hardware that may efficiently run this type of algorithm is neuromorphic hardware. It emulates the way the brain processes information and maps neurons and synapses directly into a physical substrate. Memristive devices have been identified as potential synaptic elements in neuromorphic hardware. In particular, redox-induced resistive random access memories (ReRAM) devices stand out at many aspects. They permit scalability, are energy efficient and fast, and can implement biological plasticity rules. In this work, we study the feasibility of using ReRAM devices as a replacement of the biological synapses in the sequence learning model. We implement and simulate the model including the ReRAM plasticity using the neural network simulator NEST. We investigate two types of ReRAM memristive devices: (i) a gradual, analog switching device, and (ii) an abrupt, binary switching device. We study the effect of different device properties on the performance characteristics of the sequence learning model, and demonstrate that, in contrast to many other artificial neural networks, this architecture is resilient with respect to changes in the on-off ratio and the conductance resolution, device variability, and device failure.
- 1 September 2023
- Published by IOP Publishing in Neuromorphic Computing and Engineering
- Vol. 3 (3) , 034014
Open AccessSequence learning in a spiking neuronal network with memristive synapses
Brain-inspired computing proposes a set of algorithmic principles that hold promise for advancing artificial intelligence. They endow systems with self learning capabilities, efficient energy usage, and high storage capacity. A core concept that lies at the heart of brain computation is sequence learning and prediction. This form of computation is essential for almost all our daily tasks such as movement generation, perception, and language. Understanding how the brain performs such a computation is not only important to advance neuroscience, but also to pave the way to new technological brain-inspired applications. A previously developed spiking neural network implementation of sequence prediction and recall learns complex, high-order sequences in an unsupervised manner by local, biologically inspired plasticity rules. An emerging type of hardware that may efficiently run this type of algorithm is neuromorphic hardware. It emulates the way the brain processes information and maps neurons and synapses directly into a physical substrate. Memristive devices have been identified as potential synaptic elements in neuromorphic hardware. In particular, redox-induced resistive random access memories (ReRAM) devices stand out at many aspects. They permit scalability, are energy efficient and fast, and can implement biological plasticity rules. In this work, we study the feasibility of using ReRAM devices as a replacement of the biological synapses in the sequence learning model. We implement and simulate the model including the ReRAM plasticity using the neural network simulator NEST. We investigate two types of ReRAM memristive devices: (i) a gradual, analog switching device, and (ii) an abrupt, binary switching device. We study the effect of different device properties on the performance characteristics of the sequence learning model, and demonstrate that, in contrast to many other artificial neural networks, this architecture is resilient with respect to changes in the on-off ratio and the conductance resolution, device variability, and device failure.
- Open Access
- 2 May 2023
- Published by IOP Publishing in Neuromorphic Computing and Engineering
- Vol. 3 (2) , 024002
System model of neuromorphic sequence learning on a memristive crossbar array
Machine learning models for sequence learning and processing often suffer from high energy consumption and require large amounts of training data. The brain presents more efficient solutions to how these types of tasks can be solved. While this has inspired the conception of novel brain-inspired algorithms, their realizations remain constrained to conventional von-Neumann machines. Therefore, the potential power efficiency of the algorithm cannot be exploited due to the inherent memory bottleneck of the computing architecture. Therefore, we present in this paper a dedicated hardware implementation of a biologically plausible version of the Temporal Memory component of the Hierarchical Temporal Memory concept. Our implementation is built on a memristive crossbar array and is the result of a hardware-algorithm co-design process. Rather than using the memristive devices solely for data storage, our approach leverages their specific switching dynamics to propose a formulation of the peripheral circuitry, resulting in a more efficient design. By combining a brain-like algorithm with emerging non-volatile memristive device technology we strive for maximum energy efficiency. We present simulation results on the training of complex high-order sequences and discuss how the system is able to predict in a context-dependent manner. Finally, we investigate the energy consumption during the training and conclude with a discussion of scaling prospects.
- 2 May 2023
- Published by IOP Publishing in Neuromorphic Computing and Engineering
- Vol. 3 (2) , 024002
Open AccessSystem model of neuromorphic sequence learning on a memristive crossbar array
Machine learning models for sequence learning and processing often suffer from high energy consumption and require large amounts of training data. The brain presents more efficient solutions to how these types of tasks can be solved. While this has inspired the conception of novel brain-inspired algorithms, their realizations remain constrained to conventional von-Neumann machines. Therefore, the potential power efficiency of the algorithm cannot be exploited due to the inherent memory bottleneck of the computing architecture. Therefore, we present in this paper a dedicated hardware implementation of a biologically plausible version of the Temporal Memory component of the Hierarchical Temporal Memory concept. Our implementation is built on a memristive crossbar array and is the result of a hardware-algorithm co-design process. Rather than using the memristive devices solely for data storage, our approach leverages their specific switching dynamics to propose a formulation of the peripheral circuitry, resulting in a more efficient design. By combining a brain-like algorithm with emerging non-volatile memristive device technology we strive for maximum energy efficiency. We present simulation results on the training of complex high-order sequences and discuss how the system is able to predict in a context-dependent manner. Finally, we investigate the energy consumption during the training and conclude with a discussion of scaling prospects.
- Open Access
- 11 April 2023
- Published by Association for Computing Machinery (ACM)
- p. 108-114
Demonstration of neuromorphic sequence learning on a memristive array
Sequence learning and prediction are considered principle computations performed by biological brains. Machine learning algorithms solve this type of task, but they require large amounts of training data and a substantial energy budget. An approach to overcome these issues and enable sequence learning with brain-like performance is neuromorphic hardware with brain-inspired learning algorithms. The Hierarchical Temporal Memory (HTM) is an algorithm inspired by the working principles of the neocortex and is able to learn and predict continuous sequences of elements. In a previous study, we showed that memristive devices, an emerging non-volatile memory technology, that is considered for energy efficient neuromorphic hardware, can be used as synapses in a biologically plausible version of the temporal memory algorithm of the HTM model. We subsequently presented a simulation study of an analog-mixed signal memristive hardware architecture that can implement the temporal learning algorithm. This architecture, which we refer to as MemSpikingTM, is based on a memristive crossbar array and a control circuitry implementing the neurons and the learning mechanism. In the study presented here, we demonstrate the functionality of the MemSpikingTM algorithm on a real memristive crossbar array, taped out in a commercially available 130nm CMOS technology node co-integrated with HfO based memristive devices. We explain the algorithm and the functionality of the crossbar array and peripheral circuitry and finally demonstrate context-dependent sequence learning using high-order sequences.
- 11 April 2023
- Published by Association for Computing Machinery (ACM)
- p. 108-114
Open AccessDemonstration of neuromorphic sequence learning on a memristive array
Sequence learning and prediction are considered principle computations performed by biological brains. Machine learning algorithms solve this type of task, but they require large amounts of training data and a substantial energy budget. An approach to overcome these issues and enable sequence learning with brain-like performance is neuromorphic hardware with brain-inspired learning algorithms. The Hierarchical Temporal Memory (HTM) is an algorithm inspired by the working principles of the neocortex and is able to learn and predict continuous sequences of elements. In a previous study, we showed that memristive devices, an emerging non-volatile memory technology, that is considered for energy efficient neuromorphic hardware, can be used as synapses in a biologically plausible version of the temporal memory algorithm of the HTM model. We subsequently presented a simulation study of an analog-mixed signal memristive hardware architecture that can implement the temporal learning algorithm. This architecture, which we refer to as MemSpikingTM, is based on a memristive crossbar array and a control circuitry implementing the neurons and the learning mechanism. In the study presented here, we demonstrate the functionality of the MemSpikingTM algorithm on a real memristive crossbar array, taped out in a commercially available 130nm CMOS technology node co-integrated with HfO based memristive devices. We explain the algorithm and the functionality of the crossbar array and peripheral circuitry and finally demonstrate context-dependent sequence learning using high-order sequences.