Josef Faller

Musical improvisers are trained to categorize certain musical structures into functional classes, which is thought to facilitate improvisation. Using a novel auditory oddball paradigm (Goldman et al., 2020) which enables us to disassociate a deviant (i.e. musical cord inversion) from a consistent functional class, we recorded scalp EEG from a group of musicians who spanned a range of improvisational and classically trained experience. Using a spatiospectral based inter and intra network connectivity analysis, we found that improvisers showed a variety of differences in connectivity within and between large-scale cortical networks compared to classically trained musicians, as a function of deviant type. Inter-network connectivity in the alpha band, for a time window leading up to the behavioural response, was strongly linked to improvisation experience, with the default mode network acting as a hub. Spatiospectral networks post response were substantially different between improvisers and classically trained musicians, with greater inter-network connectivity (specific to the alpha and beta bands) seen in improvisers whereas those with more classical training had largely reduced inter-network activity (mostly in the gamma band). More generally, we interpret our findings in the context of network-level correlates of expectation violation as a function of subject expertise, and we discuss how these may generalize to other and more ecologically valid scenarios.

Objective: The concurrent recording of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) is a technique that has received much attention due to its potential for combined high temporal and spatial resolution. However, the ballistocardiogram (BCG), a large-amplitude artifact caused by cardiac induced movement contaminates the EEG during EEG-fMRI recordings. Removal of BCG in software has generally made use of linear decompositions of the corrupted EEG. This is not ideal as the BCG signal propagates in a manner which is non-linearly dependent on the electrocardiogram (ECG). In this paper, we present a novel method for BCG artifact suppression using recurrent neural networks (RNNs). Methods: EEG signals were recovered by training RNNs on the nonlinear mappings between ECG and the BCG corrupted EEG. We evaluated our model's performance against the commonly used Optimal Basis Set (OBS) method at the level of individual subjects, and investigated generalization across subjects. Results: We show that our algorithm can generate larger average power reduction of the BCG at critical frequencies, while simultaneously improving task relevant EEG based classification. Conclusion: The presented deep learning architecture can be used to reduce BCG related artifacts in EEG-fMRI recordings. Significance: We present a deep learning approach that can be used to suppress the BCG artifact in EEG-fMRI without the use of additional hardware. This method may have scope to be combined with current hardware methods, operate in real-time and be used for direct modeling of the BCG.

Our state of arousal can significantly affect our ability to make optimal decisions, judgments, and actions in real-world dynamic environments. The Yerkes–Dodson law, which posits an inverse-U relationship between arousal and task performance, suggests that there is a state of arousal that is optimal for behavioral performance in a given task. Here we show that we can use online neurofeedback to shift an individual’s arousal from the right side of the Yerkes–Dodson curve to the left toward a state of improved performance. Specifically, we use a brain–computer interface (BCI) that uses information in the EEG to generate a neurofeedback signal that dynamically adjusts an individual’s arousal state when they are engaged in a boundary-avoidance task (BAT). The BAT is a demanding sensory-motor task paradigm that we implement as an aerial navigation task in virtual reality and which creates cognitive conditions that escalate arousal and quickly results in task failure (e.g., missing or crashing into the boundary). We demonstrate that task performance, measured as time and distance over which the subject can navigate before failure, is significantly increased when veridical neurofeedback is provided. Simultaneous measurements of pupil dilation and heart-rate variability show that the neurofeedback indeed reduces arousal. Our work demonstrates a BCI system that uses online neurofeedback to shift arousal state and increase task performance in accordance with the Yerkes–Dodson law.

Our state of arousal can significantly affect our ability to make optimal decisions, judgments, and actions in real-world dynamic environments. The Yerkes-Dodson law, which posits an inverse-U relationship between arousal and task performance, suggests that there is a state of arousal that is optimal for behavioral performance in a given task. Here we show that we can use on-line neurofeedback to shift an individual’s arousal toward this optimal state. Specifically, we use a brain computer interface (BCI) that uses information in the electroencephalogram (EEG) to generate a neurofeedback signal that dynamically adjusts an individual’s arousal state when they are engaged in a boundary avoidance task (BAT). The BAT is a demanding sensory-motor task paradigm that we implement as an aerial navigation task in virtual reality (VR), and which creates cognitive conditions that escalate arousal and quickly results in task failure — e.g. missing or crashing into the boundary. We demonstrate that task performance, measured as time and distance over which the subject can navigate before failure, is significantly increased when veridical neurofeedback is provided. Simultaneous measurements of pupil dilation and heart rate variability show that the neurofeedback indeed reduces arousal. Our work is the first demonstration of a BCI system that uses on-line neurofeedback to shift arousal state and increase task performance in accordance with the Yerkes-Dodson law.

State of the art brain-computer interfaces (BCIs) largely focus on detecting single, specific, often experimentally induced or manipulated aspects of the user state. In a less controlled, more naturalistic environment, a larger variety of mental processes may be active and possibly interacting. When moving BCI applications from the lab to real-life applications, these additional unaccounted mental processes could interfere with user state decoding, thus decreasing system efficacy and decreasing real-world applicability. Here, we assess the impact of affective valence on classification of working memory load, by re-analyzing a dataset that used an affective N-back task with picture stimuli. Our analyses showed that classification of working memory load under affective valence can lead to good classification accuracies (> 70%), which can be further improved via data integration over time. However, positive as well as negative affective valence resulted in decreased classification accuracies, when compared to the neutral affective context. Furthermore, classifiers failed to generalize across affective contexts, highlighting the need for user state models that can account for different contexts or new, context-independent, EEG features.

Simultaneously recorded electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) can be used to non-invasively measure the spatiotemporal dynamics of the human brain. One challenge is dealing with the artifacts that each modality introduces into the other when the two are recorded concurrently, for example the ballistocardiogram (BCG). We conducted a preliminary comparison of three different MR compatible EEG recording systems and assessed their performance in terms of single-trial classification of the EEG when simultaneously collecting fMRI. We found tradeoffs across all three systems, for example varied ease of setup and improved classification accuracy with reference electrodes (REF) but not for pulse artifact subtraction (PAS) or reference layer adaptive filtering (RLAF).