Nicholas A. Ketz
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Nicholas A. Ketz has a total of 23 co-authors. Here are the top co-authors by number of shared publications:
- Praveen Kumar Pilly
Intelligent Systems Center, HRL Laboratories, 3011 Malibu Canyon Road, Malibu, 90265, CA, USA
7 shared publications - Aaron P. Jones
Psychology Clinical Neuroscience Center, Department of Psychology, The University of New Mexico, Albuquerque, NM 87131, USA
3 shared publications - Vincent P. Clark
Department of Psychology, University of New Mexico, Albuquerque, NM 87131, USA
3 shared publications - Andrea Soltoggio
Loughborough University
2 shared publications - Bradley Robert
Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
2 shared publications - Eseoghene Ben-Iwhiwhu
Department of Computer Science Loughborough University UK
2 shared publications
Latest Publications
- Open Access
- 9 March 2023
- Published by MDPI AG in Brain Sciences
- Vol. 13 (3) , 468
Closed-Loop tACS Delivered during Slow-Wave Sleep Reduces Retroactive Interference on a Paired-Associates Learning Task
Previous studies have found a benefit of closed-loop transcranial alternating current stimulation (CL-tACS) matched to ongoing slow-wave oscillations (SWO) during sleep on memory consolidation for words in a paired associates task (PAT). Here, we examined the effects of CL-tACS in a retroactive interference PAT (ri-PAT) paradigm, where additional stimuli were presented to increase interference and reduce memory performance. Thirty-one participants were tested on a PAT before sleep, and CL-tACS was applied over the right and left DLPFC (F3 and F4) vs. mastoids for five cycles after detection of the onset of each discrete event of SWO during sleep. Participants were awoken the following morning, learned a new PAT list, and then were tested on the original list. There was a significant effect of stimulation condition (p = 0.04297; Cohen’s d = 0.768), where verum stimulation resulted in reduced retroactive interference compared with sham and a significant interaction of encoding strength and stimulation condition (p = 0.03591). Planned simple effects testing within levels of encoding revealed a significant effect of stimulation only for low-encoders (p = 0.0066; Cohen’s d = 1.075) but not high-encoders. We demonstrate here for the first time that CL-tACS during sleep can enhance the protective benefits on retroactive interference in participants who have lower encoding aptitude.
- 9 March 2023
- Published by MDPI AG in Brain Sciences
- Vol. 13 (3) , 468
Open AccessClosed-Loop tACS Delivered during Slow-Wave Sleep Reduces Retroactive Interference on a Paired-Associates Learning Task
Previous studies have found a benefit of closed-loop transcranial alternating current stimulation (CL-tACS) matched to ongoing slow-wave oscillations (SWO) during sleep on memory consolidation for words in a paired associates task (PAT). Here, we examined the effects of CL-tACS in a retroactive interference PAT (ri-PAT) paradigm, where additional stimuli were presented to increase interference and reduce memory performance. Thirty-one participants were tested on a PAT before sleep, and CL-tACS was applied over the right and left DLPFC (F3 and F4) vs. mastoids for five cycles after detection of the onset of each discrete event of SWO during sleep. Participants were awoken the following morning, learned a new PAT list, and then were tested on the original list. There was a significant effect of stimulation condition (p = 0.04297; Cohen’s d = 0.768), where verum stimulation resulted in reduced retroactive interference compared with sham and a significant interaction of encoding strength and stimulation condition (p = 0.03591). Planned simple effects testing within levels of encoding revealed a significant effect of stimulation only for low-encoders (p = 0.0066; Cohen’s d = 1.075) but not high-encoders. We demonstrate here for the first time that CL-tACS during sleep can enhance the protective benefits on retroactive interference in participants who have lower encoding aptitude.
- 9 September 2022
- Published by Springer Nature
- p. 115-126
Adapting to Environment Changes Through Neuromodulation of Reinforcement Learning
- 9 September 2022
- Published by Springer Nature
- p. 115-126
Adapting to Environment Changes Through Neuromodulation of Reinforcement Learning
- Open Access
Context Meta-Reinforcement Learning via Neuromodulation
Meta-reinforcement learning (meta-RL) algorithms enable agents to adapt quickly to tasks from few samples in dynamic environments. Such a feat is achieved through dynamic representations in an agent's policy network (obtained via reasoning about task context, model parameter updates, or both). However, obtaining rich dynamic representations for fast adaptation beyond simple benchmark problems is challenging due to the burden placed on the policy network to accommodate different policies. This paper addresses the challenge by introducing neuromodulation as a modular component to augment a standard policy network that regulates neuronal activities in order to produce efficient dynamic representations for task adaptation. The proposed extension to the policy network is evaluated across multiple discrete and continuous control environments of increasing complexity. To prove the generality and benefits of the extension in meta-RL, the neuromodulated network was applied to two state-of-the-art meta-RL algorithms (CAVIA and PEARL). The result demonstrates that meta-RL augmented with neuromodulation produces significantly better result and richer dynamic representations in comparison to the baselines.
Open AccessContext Meta-Reinforcement Learning via Neuromodulation
Meta-reinforcement learning (meta-RL) algorithms enable agents to adapt quickly to tasks from few samples in dynamic environments. Such a feat is achieved through dynamic representations in an agent's policy network (obtained via reasoning about task context, model parameter updates, or both). However, obtaining rich dynamic representations for fast adaptation beyond simple benchmark problems is challenging due to the burden placed on the policy network to accommodate different policies. This paper addresses the challenge by introducing neuromodulation as a modular component to augment a standard policy network that regulates neuronal activities in order to produce efficient dynamic representations for task adaptation. The proposed extension to the policy network is evaluated across multiple discrete and continuous control environments of increasing complexity. To prove the generality and benefits of the extension in meta-RL, the neuromodulated network was applied to two state-of-the-art meta-RL algorithms (CAVIA and PEARL). The result demonstrates that meta-RL augmented with neuromodulation produces significantly better result and richer dynamic representations in comparison to the baselines.
- Open Access
- 12 April 2022
- Published by Elsevier in Neural Networks
- Vol. 152, 70-79
Context meta-reinforcement learning via neuromodulation
- 12 April 2022
- Published by Elsevier in Neural Networks
- Vol. 152, 70-79
Open AccessContext meta-reinforcement learning via neuromodulation
- Open Access
Continual Learning Using World Models for Pseudo-Rehearsal
The utility of learning a dynamics/world model of the environment in reinforcement learning has been shown in a many ways. When using neural networks, however, these models suffer catastrophic forgetting when learned in a lifelong or continual fashion. Current solutions to the continual learning problem require experience to be segmented and labeled as discrete tasks, however, in continuous experience it is generally unclear what a sufficient segmentation of tasks would be. Here we propose a method to continually learn these internal world models through the interleaving of internally generated episodes of past experiences (i.e., pseudo-rehearsal). We show this method can sequentially learn unsupervised temporal prediction, without task labels, in a disparate set of Atari games. Empirically, this interleaving of the internally generated rollouts with the external environment's observations leads to a consistent reduction in temporal prediction loss compared to non-interleaved learning and is preserved over repeated random exposures to various tasks. Similarly, using a network distillation approach, we show that modern policy gradient based reinforcement learning algorithms can use this internal model to continually learn to optimize reward based on the world model's representation of the environment.
Open AccessContinual Learning Using World Models for Pseudo-Rehearsal
The utility of learning a dynamics/world model of the environment in reinforcement learning has been shown in a many ways. When using neural networks, however, these models suffer catastrophic forgetting when learned in a lifelong or continual fashion. Current solutions to the continual learning problem require experience to be segmented and labeled as discrete tasks, however, in continuous experience it is generally unclear what a sufficient segmentation of tasks would be. Here we propose a method to continually learn these internal world models through the interleaving of internally generated episodes of past experiences (i.e., pseudo-rehearsal). We show this method can sequentially learn unsupervised temporal prediction, without task labels, in a disparate set of Atari games. Empirically, this interleaving of the internally generated rollouts with the external environment's observations leads to a consistent reduction in temporal prediction loss compared to non-interleaved learning and is preserved over repeated random exposures to various tasks. Similarly, using a network distillation approach, we show that modern policy gradient based reinforcement learning algorithms can use this internal model to continually learn to optimize reward based on the world model's representation of the environment.
- Open Access
- 22 November 2018
- Published by MDPI AG in Brain Sciences
- Vol. 8 (12) , 204
The Benefits of Closed-Loop Transcranial Alternating Current Stimulation on Subjective Sleep Quality
Background: Poor sleep quality is a common complaint, affecting over one third of people in the United States. While sleep quality is thought to be related to slow-wave sleep (SWS), there has been little investigation to address whether modulating slow-wave oscillations (SWOs) that characterize SWS could impact sleep quality. Here we examined whether closed-loop transcranial alternating current stimulation (CL-tACS) applied during sleep impacts sleep quality and efficiency. Methods: CL-tACS was used in 21 participants delivered at the same frequency and in phase with endogenous SWOs during sleep. Sleep quality was assessed in the morning following either verum or sham control stimulation during sleep, with order counterbalanced within participants. Results: Higher sleep quality and efficiency were found after verum stimulation nights compared to control. The largest effects on sleep quality were found immediately following an adaptation night in the laboratory for which sleep quality was reduced. Conclusions: Applying CL-tACS at the same frequency and phase as endogenous SWOs may offer a novel method to improve subjective sleep quality after a night with poor quality sleep. CL-tACS might be helpful for increasing sleep quality and efficiency in otherwise healthy people, and in patients with clinical disorders that involve sleep deficits.
- 22 November 2018
- Published by MDPI AG in Brain Sciences
- Vol. 8 (12) , 204
Open AccessThe Benefits of Closed-Loop Transcranial Alternating Current Stimulation on Subjective Sleep Quality
Background: Poor sleep quality is a common complaint, affecting over one third of people in the United States. While sleep quality is thought to be related to slow-wave sleep (SWS), there has been little investigation to address whether modulating slow-wave oscillations (SWOs) that characterize SWS could impact sleep quality. Here we examined whether closed-loop transcranial alternating current stimulation (CL-tACS) applied during sleep impacts sleep quality and efficiency. Methods: CL-tACS was used in 21 participants delivered at the same frequency and in phase with endogenous SWOs during sleep. Sleep quality was assessed in the morning following either verum or sham control stimulation during sleep, with order counterbalanced within participants. Results: Higher sleep quality and efficiency were found after verum stimulation nights compared to control. The largest effects on sleep quality were found immediately following an adaptation night in the laboratory for which sleep quality was reduced. Conclusions: Applying CL-tACS at the same frequency and phase as endogenous SWOs may offer a novel method to improve subjective sleep quality after a night with poor quality sleep. CL-tACS might be helpful for increasing sleep quality and efficiency in otherwise healthy people, and in patients with clinical disorders that involve sleep deficits.
- 23 July 2018
- Published by Society for Neuroscience in The Journal of Neuroscience
- Vol. 38 (33) , 7314-7326
Closed-Loop Slow-Wave tACS Improves Sleep-Dependent Long-Term Memory Generalization by Modulating Endogenous Oscillations
Benefits in long-term memory retention and generalization have been shown to be related to sleep-dependent processes, which correlate with neural oscillations as measured by changes in electric potential. The specificity and causal role of these oscillations, however, are still poorly understood. Here, we investigated the potential for augmenting endogenous Slow-Wave (SW) oscillations in humans with closed-loop transcranial alternating current stimulation (tACS) with an aim towards enhancing the consolidation of recent experiences into long-term memory. Sixteen (three female) participants were trained pre-sleep on a Target Detection task identifying targets hidden in complex visual scenes. During post-training sleep closed-loop SW detection and stimulation was used to deliver tACS matching the phase and frequency of the dominant oscillation in the range of 0.5 to 1.2 Hz. Changes in performance were assessed the following day using test images that were identical to the training (‘Repeated9), and images generated from training scenes but with novel viewpoints (‘Generalized9). Results showed that active SW tACS during sleep enhanced the post vs. pre-sleep target detection accuracy for the Generalized images compared to sham nights, while no significant change was found for Repeated images. Using a frequency agnostic clustering approach sensitive to stimulation-induced spectral power changes in scalp EEG, this behavioral enhancement significantly correlated with both a post-stimulation increase and subsequent decrease in measured spectral power within the SW band, which in turn showed increased coupling with spindle amplitude. These results suggest that augmenting endogenous SW oscillations can enhance consolidation by specifically improving generalization over recognition or cued recall. SIGNIFICANCE STATEMENT This human study demonstrates the use of a closed-loop noninvasive brain stimulation method to enhance endogenous neural oscillations during sleep with the effect of improving consolidation of recent experiences into long-term memory. Here we show that transient slow oscillatory transcranial Alternating Current Stimulation (tACS) triggered by endogenous slow oscillations and matching their frequency and phase can increase slow-wave power and coupling with spindles. Further this increase correlates with over night improvements in generalization of recent training to facilitate performance in a target detection task. We also provide novel evidence for a tACS induced refractory period following the tACS induced increase. Here slow-wave power is temporarily reduced relative to sham stimulation, which nonetheless maintains a positive relationship with behavioral improvements.
- 23 July 2018
- Published by Society for Neuroscience in The Journal of Neuroscience
- Vol. 38 (33) , 7314-7326
Closed-Loop Slow-Wave tACS Improves Sleep-Dependent Long-Term Memory Generalization by Modulating Endogenous Oscillations
Benefits in long-term memory retention and generalization have been shown to be related to sleep-dependent processes, which correlate with neural oscillations as measured by changes in electric potential. The specificity and causal role of these oscillations, however, are still poorly understood. Here, we investigated the potential for augmenting endogenous Slow-Wave (SW) oscillations in humans with closed-loop transcranial alternating current stimulation (tACS) with an aim towards enhancing the consolidation of recent experiences into long-term memory. Sixteen (three female) participants were trained pre-sleep on a Target Detection task identifying targets hidden in complex visual scenes. During post-training sleep closed-loop SW detection and stimulation was used to deliver tACS matching the phase and frequency of the dominant oscillation in the range of 0.5 to 1.2 Hz. Changes in performance were assessed the following day using test images that were identical to the training (‘Repeated9), and images generated from training scenes but with novel viewpoints (‘Generalized9). Results showed that active SW tACS during sleep enhanced the post vs. pre-sleep target detection accuracy for the Generalized images compared to sham nights, while no significant change was found for Repeated images. Using a frequency agnostic clustering approach sensitive to stimulation-induced spectral power changes in scalp EEG, this behavioral enhancement significantly correlated with both a post-stimulation increase and subsequent decrease in measured spectral power within the SW band, which in turn showed increased coupling with spindle amplitude. These results suggest that augmenting endogenous SW oscillations can enhance consolidation by specifically improving generalization over recognition or cued recall. SIGNIFICANCE STATEMENT This human study demonstrates the use of a closed-loop noninvasive brain stimulation method to enhance endogenous neural oscillations during sleep with the effect of improving consolidation of recent experiences into long-term memory. Here we show that transient slow oscillatory transcranial Alternating Current Stimulation (tACS) triggered by endogenous slow oscillations and matching their frequency and phase can increase slow-wave power and coupling with spindles. Further this increase correlates with over night improvements in generalization of recent training to facilitate performance in a target detection task. We also provide novel evidence for a tACS induced refractory period following the tACS induced increase. Here slow-wave power is temporarily reduced relative to sham stimulation, which nonetheless maintains a positive relationship with behavioral improvements.