NKNicholas A. Ketz

Nicholas A. Ketz

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Nicholas A. Ketz has a total of 23 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Closed-Loop tACS Delivered during Slow-Wave Sleep Reduces Retroactive Interference on a Paired-Associates Learning Task

    by Aaron P. Jones, Natalie B. Bryant, Bradley M. Robert, Teagan S. Mullins, Michael C. S. Trumbo, Nicholas A. Ketz, Michael D. Howard, Praveen K. Pilly, Vincent P. Clark

    Previous studies have found a benefit of closed-loop transcranial alternating current stimulation (CL-tACS) matched to ongoing slow-wave oscillations (SWO) during sleep on memory consolidation for words in a paired associates task (PAT). Here, we examined the effects of CL-tACS in a retroactive interference PAT (ri-PAT) paradigm, where additional stimuli were presented to increase interference and reduce memory performance. Thirty-one participants were tested on a PAT before sleep, and CL-tACS was applied over the right and left DLPFC (F3 and F4) vs. mastoids for five cycles after detection of the onset of each discrete event of SWO during sleep. Participants were awoken the following morning, learned a new PAT list, and then were tested on the original list. There was a significant effect of stimulation condition (p = 0.04297; Cohen’s d = 0.768), where verum stimulation resulted in reduced retroactive interference compared with sham and a significant interaction of encoding strength and stimulation condition (p = 0.03591). Planned simple effects testing within levels of encoding revealed a significant effect of stimulation only for low-encoders (p = 0.0066; Cohen’s d = 1.075) but not high-encoders. We demonstrate here for the first time that CL-tACS during sleep can enhance the protective benefits on retroactive interference in participants who have lower encoding aptitude.


  • Adapting to Environment Changes Through Neuromodulation of Reinforcement Learning

    by Jinwei Xing, Xinyun Zou, Praveen K. Pilly, Nicholas A. Ketz, Jeffrey L. Krichmar


  • Open Access

    Context Meta-Reinforcement Learning via Neuromodulation

    by Eseoghene Ben-Iwhiwhu, Jeffery Dick, Nicholas A. Ketz, Praveen K. Pilly, Andrea Soltoggio

    Meta-reinforcement learning (meta-RL) algorithms enable agents to adapt quickly to tasks from few samples in dynamic environments. Such a feat is achieved through dynamic representations in an agent's policy network (obtained via reasoning about task context, model parameter updates, or both). However, obtaining rich dynamic representations for fast adaptation beyond simple benchmark problems is challenging due to the burden placed on the policy network to accommodate different policies. This paper addresses the challenge by introducing neuromodulation as a modular component to augment a standard policy network that regulates neuronal activities in order to produce efficient dynamic representations for task adaptation. The proposed extension to the policy network is evaluated across multiple discrete and continuous control environments of increasing complexity. To prove the generality and benefits of the extension in meta-RL, the neuromodulated network was applied to two state-of-the-art meta-RL algorithms (CAVIA and PEARL). The result demonstrates that meta-RL augmented with neuromodulation produces significantly better result and richer dynamic representations in comparison to the baselines.

  • Open Access

    Context meta-reinforcement learning via neuromodulation

    by Eseoghene Ben-Iwhiwhu, Jeffery Dick, Nicholas A. Ketz, Praveen K. Pilly, Andrea Soltoggio


  • Open Access

    Continual Learning Using World Models for Pseudo-Rehearsal

    by Nicholas Ketz, Soheil Kolouri, Praveen Pilly

    The utility of learning a dynamics/world model of the environment in reinforcement learning has been shown in a many ways. When using neural networks, however, these models suffer catastrophic forgetting when learned in a lifelong or continual fashion. Current solutions to the continual learning problem require experience to be segmented and labeled as discrete tasks, however, in continuous experience it is generally unclear what a sufficient segmentation of tasks would be. Here we propose a method to continually learn these internal world models through the interleaving of internally generated episodes of past experiences (i.e., pseudo-rehearsal). We show this method can sequentially learn unsupervised temporal prediction, without task labels, in a disparate set of Atari games. Empirically, this interleaving of the internally generated rollouts with the external environment's observations leads to a consistent reduction in temporal prediction loss compared to non-interleaved learning and is preserved over repeated random exposures to various tasks. Similarly, using a network distillation approach, we show that modern policy gradient based reinforcement learning algorithms can use this internal model to continually learn to optimize reward based on the world model's representation of the environment.

  • Open Access

    The Benefits of Closed-Loop Transcranial Alternating Current Stimulation on Subjective Sleep Quality

    by Charles S. H. Robinson, Natalie B. Bryant, Joshua W. Maxwell, Aaron P. Jones, Bradley Robert, Melanie Lamphere, Angela Combs, Hussein M. Al Azzawi, Benjamin C. Gibson, Joseph L. Sanguinetti, et al.

    Background: Poor sleep quality is a common complaint, affecting over one third of people in the United States. While sleep quality is thought to be related to slow-wave sleep (SWS), there has been little investigation to address whether modulating slow-wave oscillations (SWOs) that characterize SWS could impact sleep quality. Here we examined whether closed-loop transcranial alternating current stimulation (CL-tACS) applied during sleep impacts sleep quality and efficiency. Methods: CL-tACS was used in 21 participants delivered at the same frequency and in phase with endogenous SWOs during sleep. Sleep quality was assessed in the morning following either verum or sham control stimulation during sleep, with order counterbalanced within participants. Results: Higher sleep quality and efficiency were found after verum stimulation nights compared to control. The largest effects on sleep quality were found immediately following an adaptation night in the laboratory for which sleep quality was reduced. Conclusions: Applying CL-tACS at the same frequency and phase as endogenous SWOs may offer a novel method to improve subjective sleep quality after a night with poor quality sleep. CL-tACS might be helpful for increasing sleep quality and efficiency in otherwise healthy people, and in patients with clinical disorders that involve sleep deficits.


  • Closed-Loop Slow-Wave tACS Improves Sleep-Dependent Long-Term Memory Generalization by Modulating Endogenous Oscillations

    by Nicholas Ketz, Aaron P. Jones, Natalie B. Bryant, Vincent P. Clark, Praveen K. Pilly

    Benefits in long-term memory retention and generalization have been shown to be related to sleep-dependent processes, which correlate with neural oscillations as measured by changes in electric potential. The specificity and causal role of these oscillations, however, are still poorly understood. Here, we investigated the potential for augmenting endogenous Slow-Wave (SW) oscillations in humans with closed-loop transcranial alternating current stimulation (tACS) with an aim towards enhancing the consolidation of recent experiences into long-term memory. Sixteen (three female) participants were trained pre-sleep on a Target Detection task identifying targets hidden in complex visual scenes. During post-training sleep closed-loop SW detection and stimulation was used to deliver tACS matching the phase and frequency of the dominant oscillation in the range of 0.5 to 1.2 Hz. Changes in performance were assessed the following day using test images that were identical to the training (‘Repeated9), and images generated from training scenes but with novel viewpoints (‘Generalized9). Results showed that active SW tACS during sleep enhanced the post vs. pre-sleep target detection accuracy for the Generalized images compared to sham nights, while no significant change was found for Repeated images. Using a frequency agnostic clustering approach sensitive to stimulation-induced spectral power changes in scalp EEG, this behavioral enhancement significantly correlated with both a post-stimulation increase and subsequent decrease in measured spectral power within the SW band, which in turn showed increased coupling with spindle amplitude. These results suggest that augmenting endogenous SW oscillations can enhance consolidation by specifically improving generalization over recognition or cued recall. SIGNIFICANCE STATEMENT This human study demonstrates the use of a closed-loop noninvasive brain stimulation method to enhance endogenous neural oscillations during sleep with the effect of improving consolidation of recent experiences into long-term memory. Here we show that transient slow oscillatory transcranial Alternating Current Stimulation (tACS) triggered by endogenous slow oscillations and matching their frequency and phase can increase slow-wave power and coupling with spindles. Further this increase correlates with over night improvements in generalization of recent training to facilitate performance in a target detection task. We also provide novel evidence for a tACS induced refractory period following the tACS induced increase. Here slow-wave power is temporarily reduced relative to sham stimulation, which nonetheless maintains a positive relationship with behavioral improvements.