Marko Sarlija

Background/Objectives: Slow oscillation (SO) brainwaves observed during sleep have been shown to reflect the process of memory consolidation, that underlies the critical role of sleep in learning, memory, and other cognitive functions. Closed-loop auditory stimulation (CLAS) uses tones presented in phase with SOs to increase their amplitude and number, along with other brainwave signatures related to memory consolidation. Prior studies have found that CLAS maximizes the ability to perform rote memorization tasks, although this remains controversial. The present study examined whether CLAS affects a broader range of learning tasks than has been tested previously, including a rote language learning task requiring basic memorization and also two discovery learning tasks requiring insight, hypothesis testing, and integration of experience, all processes that benefit from memory consolidation. Methods: Twenty-eight healthy participants performed language and discovery learning tasks before sleeping in our laboratory for three continuous nights per week over two weeks, with verum or control CLAS using a prototype NeuroGevity system (NeuroGeneces, Inc., Santa Fe, NM, USA) in a crossed, randomized, double-blind manner. Results: Language learning showed a 35% better word recall (p = 0.048), and discovery learning showed a 26% better performance (p < 0.001) after three continuous nights of CLAS vs. control. EEG measures showed increased SO amplitude and entrainment, SO-spindle coupling, and other features that may underlie the learning benefits of CLAS. Conclusions: Taken together, the present results show that CLAS can alter brain dynamics and enhance learning, especially in complex discovery learning tasks that may benefit more from memory consolidation compared with rote word pair or language learning.

The COVID-19 pandemic has adverse consequences on human psychology and behavior long after initial recovery from the virus. These COVID-19 health sequelae, if undetected and left untreated, may lead to more enduring mental health problems, and put vulnerable individuals at risk of developing more serious psychopathologies. Therefore, an early distinction of such vulnerable individuals from those who are more resilient is important to undertake timely preventive interventions. The main aim of this article is to present a comprehensive multimodal conceptual approach for addressing these potential psychological and behavioral mental health changes using state-of-the-art tools and means of artificial intelligence (AI). Mental health COVID-19 recovery programs at post-COVID clinics based on AI prediction and prevention strategies may significantly improve the global mental health of ex-COVID-19 patients. Most COVID-19 recovery programs currently involve specialists such as pulmonologists, cardiologists, and neurologists, but there is a lack of psychiatrist care. The focus of this article is on new tools which can enhance the current limited psychiatrist resources and capabilities in coping with the upcoming challenges related to widespread mental health disorders. Patients affected by COVID-19 are more vulnerable to psychological and behavioral changes than non-COVID populations and therefore they deserve careful clinical psychological screening in post-COVID clinics. However, despite significant advances in research, the pace of progress in prevention of psychiatric disorders in these patients is still insufficient. Current approaches for the diagnosis of psychiatric disorders largely rely on clinical rating scales, as well as self-rating questionnaires that are inadequate for comprehensive assessment of ex-COVID-19 patients’ susceptibility to mental health deterioration. These limitations can presumably be overcome by applying state-of-the-art AI-based tools in diagnosis, prevention, and treatment of psychiatric disorders in acute phase of disease to prevent more chronic psychiatric consequences.

Cognitive load is related to the amount of working memory resources used in the execution of various mental tasks. Different multimodal features extracted from peripheral physiology, brain activity, and oculometric reactions have been used as non-intrusive, reliable, and objective measures of cognitive load. In this paper, we use data from 38 participants performing a four-level difficulty n-back task (0-, 1-, 2-, and 3-back task), with their oculometric reactions simultaneously recorded. Based on the neuroanatomic structure and function of the visual system, 26 oculometric features are extracted and organized into 3 groups related to: pupil dilation (PD), blinking, and fixation. The discriminative power of each group of features was evaluated in four-level cognitive load classification using a support vector machine (SVM) model and feature selection, and the achieved classification accuracies were: 33.33% using only pupil dilation features, 30.90% using only blink-related features, 30.21% using only fixations-related features. Finally, a 36.11% classification accuracy was achieved using a combination of all extracted oculometric features. The presented results show that various groups of oculometric features provide complementary information about the subject's cognitive load. The comparison of the extracted groups of features is given, and the most important features in terms of classification performance are discussed.

Cognitive load can be estimated using individuals’ task performance, their subjective measures, and neurophysiological measures. Neurophysiological measures, which among others include brain activation signals obtained with various brain imaging techniques, such as the functional near-infrared spectroscopy (fNIRS), and signals from the peripheral physiology, such as the electrocardiography (ECG) signal, allow an objective and continuous estimation of cognitive load. In this paper, the fNIRS and ECG signals were simultaneously collected from 32 participants and used to classify three levels of cognitive load on n-back task. A set of 30 fNIRS and ECG features proposed in this paper enables the classification of different levels of cognitive load on n-back task using the support vector machine (SVM), k-nearest neighbors (KNN), and linear discriminant analysis (LDA) classification models. When combining the fNIRS and ECG features, three difficulties of the n-back task were classified with the mean accuracies ranging from 61% to 67%, while two difficulties were classified with the mean accuracy ranging from 70% to 84%. The most important features in the classification are discussed. The results presented in this paper extend the existing empirical evidence that combining brain imaging and peripheral physiology features increases the accuracy of multi-level cognitive load classification, thus further underscoring the importance of multimodal approach to cognitive load classification.

In this paper we investigate the potential of generic physiological features of stress resilience in predicting air traffic control (ATC) candidates' performance in a highly-stressful low-fidelity ATC simulator scenario. Stress resilience is highlighted as an important occupational factor that influences the performance and well-being of air traffic control officers (ATCO). Poor stress management, besides the lack of skills, can be a direct cause of poor performance under stress, both in the selection process of ATCOs and later in the workplace. 40 ATC candidates, within the final stages of their selection process, underwent a stimulation paradigm for elicitation and assessment of various generic task-unrelated physiological features, related to resting heart rate variability (HRV) and respiratory sinus arrhythmia (RSA), acoustic startle response (ASR) and the physiological allostatic response, which are all recognized as relevant psychophysiological markers of stress resilience. The multimodal approach included analysis of electrocardiography, electromyography, electrodermal activity and respiration. We make advances in computational methodology for assessment of physiological features of stress resilience, and investigate the predictive power of the obtained feature space in a binary classification problem: prediction of high- vs. low-performance on the developed ATC simulator. Our novel approach yields a relatively high 78.16% classification accuracy. These results are discussed in the context of prior work, while considering study limitations and proposing directions for future work.

Comprehensive multimodal psychophysiological measurements and smart data analysis based on wearable and low-cost technologies could enhance traditional air traffic controller (ATC) selection process. Many recent studies in neuro-cognitive science and stress resilience illustrated effectiveness of these multimodal measurements and appropriate metrics in comprehensive assessment of ATCs’ mental states, such as cognitive workload, cognitive decline, attention deficit, fatigue, emotional and behavioural problems, etc. Accordingly, this article is focused on innovation efforts in ATC selection protocols based on a set of comprehensive stimuli and corresponding multimodal psychophysiological measurements. The concept of enhancement of ATC selection process presented in this article includes complex physiological, oculometric and speech measurements and appropriate metrics. From these multimodal measurements during specific stimulation tasks, which include different versions of acoustic startle stimuli, airblasts, semantically relevant aversive images and sounds, different versions of Stroop tests, visual tracking test, a complex set of multimodal-multidimensional features is computed as predictors of ATC candidates’ future performance, like: stress resilience, workload capacity, attention, visual performance, working memory etc. Such cost-effective, more objective, non-invasive preliminary measurements, lasting no longer than 45 minutes may have good discriminative power and might be used in ATC selection processes as enhancement of current selection procedures. Comprehensive analysis of presented multimodal features during different experimental conditions might also be very useful in selection processes of other stressful professional jobs, like first responders, pilots, astronauts etc.