Hrishikesh M. Rao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Hrishikesh M. Rao has a total of 23 co-authors. Here are the top co-authors by number of shared publications:
- Christopher J. Smalt
Human Health and Performance Systems Group, MIT Lincoln Laboratory, Lexington, MA, USA
2 shared publications - Stephanie Haro
Human Health and Performance Systems Group, MIT Lincoln Laboratory, Lexington, MA, USA
2 shared publications - Thomas Francis Quatieri
MIT Lincoln Laboratory
2 shared publications - Chancellor Johnstone
Department of Mathematics Statistics, Air Force Institute of Technology, Wright-Patterson Air Force Base, OH, USA
1 shared publication - Christine Beauchene
Human Health and Performance Systems Group, MIT Lincoln Laboratory, Lexington, MA, USA
1 shared publication Christopher J Smalt
Human Health & Performance Systems Group, MIT Lincoln Laboratory , Lexington, MA 02420 USA
1 shared publication
Latest Publications
- 29 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 56 (1) , 59-73
Heart Rate Varies With Mental Workload and Performance in Virtual Reality Flight Tasks
Heart rate (HR) has been shown to correlate with mental workload in operational and instructional settings, supporting its use as an objective, continuous, and unobtrusive workload index. With the growing use of virtual reality (VR) in training and education, we evaluate whether the HR–workload relationship holds in VR simulation. To this end, we analyze HR responses derived from electrocardiogram (ECG) recordings in 20 participants performing VR flight maneuvers and assess the relationship between HR, mental workload, and task performance. Our analysis reveals that mean HR increases by 2% ( $p \lt 0.05$ ) across task difficulty levels, decreases by 5% ( $p \lt 0.01$ ) across repeated runs of the same difficulty level, and increases by 3% ( $p \lt 0.01$ ) near touchdown versus the start of the flight. We find that HR correlates positively with mental workload and negatively with task performance and that HR provides insights into mental workload beyond what performance alone reveals. We also find that HR in our VR task follows expected mental workload-related trends consistent with observations from other environments (real aircraft and legacy flight simulators), and that photoplethysmography (PPG)-derived HR closely tracked ECG patterns, supporting unobtrusive monitoring in VR. Our findings suggest that HR is a readily measurable, meaningful signal for tracking mental workload in VR and could enable real-time, closed-loop personalization of training difficulty to maintain optimal engagement.
- 29 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 56 (1) , 59-73
Heart Rate Varies With Mental Workload and Performance in Virtual Reality Flight Tasks
Heart rate (HR) has been shown to correlate with mental workload in operational and instructional settings, supporting its use as an objective, continuous, and unobtrusive workload index. With the growing use of virtual reality (VR) in training and education, we evaluate whether the HR–workload relationship holds in VR simulation. To this end, we analyze HR responses derived from electrocardiogram (ECG) recordings in 20 participants performing VR flight maneuvers and assess the relationship between HR, mental workload, and task performance. Our analysis reveals that mean HR increases by 2% ( $p \lt 0.05$ ) across task difficulty levels, decreases by 5% ( $p \lt 0.01$ ) across repeated runs of the same difficulty level, and increases by 3% ( $p \lt 0.01$ ) near touchdown versus the start of the flight. We find that HR correlates positively with mental workload and negatively with task performance and that HR provides insights into mental workload beyond what performance alone reveals. We also find that HR in our VR task follows expected mental workload-related trends consistent with observations from other environments (real aircraft and legacy flight simulators), and that photoplethysmography (PPG)-derived HR closely tracked ECG patterns, supporting unobtrusive monitoring in VR. Our findings suggest that HR is a readily measurable, meaningful signal for tracking mental workload in VR and could enable real-time, closed-loop personalization of training difficulty to maintain optimal engagement.
- 30 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive and Developmental Systems
- Vol. 18 (1) , 214-227
Student-Pilot Error Prediction via Multimodal Physiological Signals and Tree-Based Models
The United States Air Force (USAF) faces significant challenges in training new pilots to meet the growing demand for skilled airmen in various essential operations. While varied in nature, these challenges are underpinned by significant resource constraints, e.g., limited trainer aircraft ability, shortage of instructor pilots, and weather-related limitations. Recent USAF initiatives leverage simulator flights in training programs to ameliorate such issues; however, given that the USAF also faces a lack of available simulator instructors, resource limitations are not entirely circumvented. Therefore, this study aims to alleviate the strain on instructors by building a machine learning pipeline to predict flight errors in simulator flights. By accurately quantifying errors, the burden of the time-consuming duties of instructors can be reduced, allowing them to focus instead on personalized training for student pilots. Leveraging physiological and aircraft data recorded during simulator flights involving aircraft landings, this research uses functional data analysis, modern feature-selection techniques, and efficient tree-based machine learning methods to predict student performance and highlight key predictive physiological modalities. The proposed pipeline, coined the Multimodal Functional Learning for Error Regression (mFLYER), produces various models that demonstrate notable improvements over prior approaches, closing the gap between successful manual and automated grades. Finally, to further examine the applicability of such models, this approach leverages uncertainty quantification through calibrated models, then ensembles, to identify their ability to confidently differentiate between low- and high-error landings, which many models struggle to accomplish. Finally, additional analysis is performed to determine how future research may improve model performance and confidence.
- 30 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive and Developmental Systems
- Vol. 18 (1) , 214-227
Student-Pilot Error Prediction via Multimodal Physiological Signals and Tree-Based Models
The United States Air Force (USAF) faces significant challenges in training new pilots to meet the growing demand for skilled airmen in various essential operations. While varied in nature, these challenges are underpinned by significant resource constraints, e.g., limited trainer aircraft ability, shortage of instructor pilots, and weather-related limitations. Recent USAF initiatives leverage simulator flights in training programs to ameliorate such issues; however, given that the USAF also faces a lack of available simulator instructors, resource limitations are not entirely circumvented. Therefore, this study aims to alleviate the strain on instructors by building a machine learning pipeline to predict flight errors in simulator flights. By accurately quantifying errors, the burden of the time-consuming duties of instructors can be reduced, allowing them to focus instead on personalized training for student pilots. Leveraging physiological and aircraft data recorded during simulator flights involving aircraft landings, this research uses functional data analysis, modern feature-selection techniques, and efficient tree-based machine learning methods to predict student performance and highlight key predictive physiological modalities. The proposed pipeline, coined the Multimodal Functional Learning for Error Regression (mFLYER), produces various models that demonstrate notable improvements over prior approaches, closing the gap between successful manual and automated grades. Finally, to further examine the applicability of such models, this approach leverages uncertainty quantification through calibrated models, then ensembles, to identify their ability to confidently differentiate between low- and high-error landings, which many models struggle to accomplish. Finally, additional analysis is performed to determine how future research may improve model performance and confidence.
- Open Access
- 16 May 2023
- Published by Oxford University Press (OUP) in Military Medicine
- Vol. 188 (11-12) , e3398-e3406
Changes in Eye Tracking Features Across Periods of Overpressure Exposure
Repetitive exposure to blast overpressure waves can be a part of routine military and law enforcement training. However, our understanding of the effects of that repetitive exposure on human neurophysiology remains limited. To link an individual’s cumulative exposure with their neurophysiological effects, overpressure dosimetry needs to be concurrently collected with relevant physiological signals. Eye tracking has shown promise for providing insight into neurophysiological change because of neural injury, but video-based technology limits usage to a laboratory or clinic. In the present work, we show capability for using electrooculography-based eye tracking to enable physiological assessment in the field during activities involved repetitive blast exposures. Overpressure dosimetry was accomplished by using a body-worn measurement system that captures continuous sound pressure levels as well as pressure waveforms of blast event in the range of 135-185 dB peak (0.1-36 kPa). Electrooculography eye tracking was performed using a commercial Shimmer Sensing system, which captured horizontal eye movements of both the left and right eyes, as well as vertical eye movements of the right eye, from which blinks can also be extracted. Data were collected during breaching activities that included repetitive use of explosives. Participants in the study were U.S. Army Special Operators and Federal Bureau of Investigations special agents. Approval for research was received by the Massachucetts Institute of Technology Committee on the Use of Humans as Experimental Subjects, the Air Force Human Research Protections Office, and the Federal Bureau of Investigations Institutional Review Board. The energy from overpressure events was accumulated and summarized into an 8-hour equivalent of sound pressure level (i.e., LZeq8hr). The total exposure in a single day, i.e., the LZeq8hr, ranged from 110 to 160 dB. Oculomotor features, such as blink and saccade rate, as well as variance in blink waveforms, show changes across the period of overpressure exposure. However, the features that showed significant change across the population were not necessarily the ones that showed significant correlation with the levels of overpressure exposure. A regression model built to predict overpressure levels from oculomotor features alone showed a significant association (R = 0.51, P < .01). Investigation of the model indicates that changes in the saccade rate and blink waveforms are driving the relationship. This study successfully demonstrated that eye tracking can be performed during training activities, such as explosive breaching, and that the modality may provide insight into neurophysiological change across periods of overpressure exposure. The results presented herein show that electrooculography-based eye tracking may be a useful method of assessing individualized physiological effects of overpressure exposure in the field. Future work is focused on time-dependent modeling to assess continuous changes in eye movements as this will enable building dose–response curves.
- 16 May 2023
- Published by Oxford University Press (OUP) in Military Medicine
- Vol. 188 (11-12) , e3398-e3406
Open AccessChanges in Eye Tracking Features Across Periods of Overpressure Exposure
Repetitive exposure to blast overpressure waves can be a part of routine military and law enforcement training. However, our understanding of the effects of that repetitive exposure on human neurophysiology remains limited. To link an individual’s cumulative exposure with their neurophysiological effects, overpressure dosimetry needs to be concurrently collected with relevant physiological signals. Eye tracking has shown promise for providing insight into neurophysiological change because of neural injury, but video-based technology limits usage to a laboratory or clinic. In the present work, we show capability for using electrooculography-based eye tracking to enable physiological assessment in the field during activities involved repetitive blast exposures. Overpressure dosimetry was accomplished by using a body-worn measurement system that captures continuous sound pressure levels as well as pressure waveforms of blast event in the range of 135-185 dB peak (0.1-36 kPa). Electrooculography eye tracking was performed using a commercial Shimmer Sensing system, which captured horizontal eye movements of both the left and right eyes, as well as vertical eye movements of the right eye, from which blinks can also be extracted. Data were collected during breaching activities that included repetitive use of explosives. Participants in the study were U.S. Army Special Operators and Federal Bureau of Investigations special agents. Approval for research was received by the Massachucetts Institute of Technology Committee on the Use of Humans as Experimental Subjects, the Air Force Human Research Protections Office, and the Federal Bureau of Investigations Institutional Review Board. The energy from overpressure events was accumulated and summarized into an 8-hour equivalent of sound pressure level (i.e., LZeq8hr). The total exposure in a single day, i.e., the LZeq8hr, ranged from 110 to 160 dB. Oculomotor features, such as blink and saccade rate, as well as variance in blink waveforms, show changes across the period of overpressure exposure. However, the features that showed significant change across the population were not necessarily the ones that showed significant correlation with the levels of overpressure exposure. A regression model built to predict overpressure levels from oculomotor features alone showed a significant association (R = 0.51, P < .01). Investigation of the model indicates that changes in the saccade rate and blink waveforms are driving the relationship. This study successfully demonstrated that eye tracking can be performed during training activities, such as explosive breaching, and that the modality may provide insight into neurophysiological change across periods of overpressure exposure. The results presented herein show that electrooculography-based eye tracking may be a useful method of assessing individualized physiological effects of overpressure exposure in the field. Future work is focused on time-dependent modeling to assess continuous changes in eye movements as this will enable building dose–response curves.
- Open Access
- 16 February 2022
- Published by Wiley in European Journal of Neuroscience
- Vol. 55 (5) , 1262-1277
EEG alpha and pupil diameter reflect endogenous auditory attention switching and listening effort
Everyday environments often contain distracting competing talkers and background noise, requiring listeners to focus their attention on one acoustic source and reject others. During this auditory attention task, listeners may naturally interrupt their sustained attention and switch attended sources. The effort required to perform this attention switch has not been well studied in the context of competing continuous speech. In this work, we developed two variants of endogenous attention switching and a sustained attention control. We characterized these three experimental conditions under the context of decoding auditory attention, while simultaneously evaluating listening effort and neural markers of spatial‐audio cues. A least‐squares, electroencephalography (EEG)‐based, attention decoding algorithm was implemented across all conditions. It achieved an accuracy of 69.4% and 64.0% when computed over nonoverlapping 10 and 5‐s correlation windows, respectively. Both decoders illustrated smooth transitions in the attended talker prediction through switches at approximately half of the analysis window size (e.g., the mean lag taken across the two switch conditions was 2.2 s when the 5‐s correlation window was used). Expended listening effort, as measured by simultaneous EEG and pupillometry, was also a strong indicator of whether the listeners sustained attention or performed an endogenous attention switch (peak pupil diameter measure [ ] and minimum parietal alpha power measure [ ]). We additionally found evidence of talker spatial cues in the form of centrotemporal alpha power lateralization ( ). These results suggest that listener effort and spatial cues may be promising features to pursue in a decoding context, in addition to speech‐based features.
- 16 February 2022
- Published by Wiley in European Journal of Neuroscience
- Vol. 55 (5) , 1262-1277
Open AccessEEG alpha and pupil diameter reflect endogenous auditory attention switching and listening effort
Everyday environments often contain distracting competing talkers and background noise, requiring listeners to focus their attention on one acoustic source and reject others. During this auditory attention task, listeners may naturally interrupt their sustained attention and switch attended sources. The effort required to perform this attention switch has not been well studied in the context of competing continuous speech. In this work, we developed two variants of endogenous attention switching and a sustained attention control. We characterized these three experimental conditions under the context of decoding auditory attention, while simultaneously evaluating listening effort and neural markers of spatial‐audio cues. A least‐squares, electroencephalography (EEG)‐based, attention decoding algorithm was implemented across all conditions. It achieved an accuracy of 69.4% and 64.0% when computed over nonoverlapping 10 and 5‐s correlation windows, respectively. Both decoders illustrated smooth transitions in the attended talker prediction through switches at approximately half of the analysis window size (e.g., the mean lag taken across the two switch conditions was 2.2 s when the 5‐s correlation window was used). Expended listening effort, as measured by simultaneous EEG and pupillometry, was also a strong indicator of whether the listeners sustained attention or performed an endogenous attention switch (peak pupil diameter measure [ ] and minimum parietal alpha power measure [ ]). We additionally found evidence of talker spatial cues in the form of centrotemporal alpha power lateralization ( ). These results suggest that listener effort and spatial cues may be promising features to pursue in a decoding context, in addition to speech‐based features.
- Open Access
EEG Alpha Power and Pupil Diameter Reflect Endogenous Auditory Attention Switching and Listening Effort
Auditory attention describes a listeners focus on an acoustic source while they ignore other competing sources that might be present. In an environment with multiple talkers and background noise (i.e. the cocktail party effect), auditory attention can be difficult, requiring the listener to expend measurable cognitive effort. A listener will naturally interrupt sustained attention on a source when switching towards another source during conversation. This change in attention is potentially even more taxing than maintaining sustained attention due to the limits of human working memory, and this additional effort required has not been well studied. In this work, we evaluated an attention decoder algorithm for detecting the change in attention and investigated cognitive effort expended during attentional switching and sustained attention. Two variants of endogenous attention switching were explored: the switches either had in-the-moment decision making or a pre-defined attentional switch time. A least-squares, EEG-based, attention decoding algorithm achieved 64.1% accuracy with a 5-second correlation window and illustrated smooth transitions in the attended talker prediction through switches in sustained attention at approximately half of the analysis window size (2.2 seconds). The expended listening effort, as measured by simultaneous electroencephalography (EEG) and pupillometry, was also a strong indicator of switching. Specifically, centrotemporal alpha power [F(2, 18) = 7.473, P = 0.00434] and mean pupil diameter [F(2, 18) = 9.159, P = 0.0018] were significantly different for trials that contained a switch in comparison to sustained trials. We also found that relative attended and ignored talker locations modulate the EEG alpha topographic response. This alpha lateralization was found to be impacted by the interaction between experimental condition and whether the measure was computed before or after the switch [F(2,18) = 3.227, P = 0.0634]. These results suggest that expended listening effort is a promising feature that should be pursued in a decoding context, in addition to speech and location-based features.
Open AccessEEG Alpha Power and Pupil Diameter Reflect Endogenous Auditory Attention Switching and Listening Effort
Auditory attention describes a listeners focus on an acoustic source while they ignore other competing sources that might be present. In an environment with multiple talkers and background noise (i.e. the cocktail party effect), auditory attention can be difficult, requiring the listener to expend measurable cognitive effort. A listener will naturally interrupt sustained attention on a source when switching towards another source during conversation. This change in attention is potentially even more taxing than maintaining sustained attention due to the limits of human working memory, and this additional effort required has not been well studied. In this work, we evaluated an attention decoder algorithm for detecting the change in attention and investigated cognitive effort expended during attentional switching and sustained attention. Two variants of endogenous attention switching were explored: the switches either had in-the-moment decision making or a pre-defined attentional switch time. A least-squares, EEG-based, attention decoding algorithm achieved 64.1% accuracy with a 5-second correlation window and illustrated smooth transitions in the attended talker prediction through switches in sustained attention at approximately half of the analysis window size (2.2 seconds). The expended listening effort, as measured by simultaneous electroencephalography (EEG) and pupillometry, was also a strong indicator of switching. Specifically, centrotemporal alpha power [F(2, 18) = 7.473, P = 0.00434] and mean pupil diameter [F(2, 18) = 9.159, P = 0.0018] were significantly different for trials that contained a switch in comparison to sustained trials. We also found that relative attended and ignored talker locations modulate the EEG alpha topographic response. This alpha lateralization was found to be impacted by the interaction between experimental condition and whether the measure was computed before or after the switch [F(2,18) = 3.227, P = 0.0634]. These results suggest that expended listening effort is a promising feature that should be pursued in a decoding context, in addition to speech and location-based features.
- Open Access
- 1 March 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 48, 133-140
Evaluating the effects of image persistence on dynamic target acquisition in low frame rate virtual environments
User performance in virtual environments with degraded visual conditions due to low frame rates is an interesting area of inquiry. Visual content shown in a low frame rate simulation has the quality of the original image, but persists for an extended period until the next frame is displayed (so-called high persistence-HP). An alternative, called low persistence (LP), involves displaying the rendered frame for a single display frame and blanking the screen while waiting for the next frame to be generated. Previous research has evaluated the usefulness of the LP technique in low frame rate simulations during a static target acquisition task. To gain greater knowledge about the LP technique, we have conducted a user study to evaluate user performance and learning during a dynamic target acquisition task. The acquisition task was evaluated under a high frame rate, (60 fps) condition, a traditional low frame rate HP condition (10 fps), and the experimental low frame rate LP technique. The task involved the acquisition of targets moving along several different trajectories, modeled after a shotgun trap shooting task. The results of our study indicate the LP condition approaches high frame rate performance within certain classes of target trajectories. Interestingly we also see that learning is consistent across conditions, indicating that it may not always be necessary to train under a visually high frame rate system to learn a particular task. We discuss implications of using the LP technique to mitigate low frame rate issues as well as its potential usefulness for training in low frame rate virtual environments.
- 1 March 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 48, 133-140
Open AccessEvaluating the effects of image persistence on dynamic target acquisition in low frame rate virtual environments
User performance in virtual environments with degraded visual conditions due to low frame rates is an interesting area of inquiry. Visual content shown in a low frame rate simulation has the quality of the original image, but persists for an extended period until the next frame is displayed (so-called high persistence-HP). An alternative, called low persistence (LP), involves displaying the rendered frame for a single display frame and blanking the screen while waiting for the next frame to be generated. Previous research has evaluated the usefulness of the LP technique in low frame rate simulations during a static target acquisition task. To gain greater knowledge about the LP technique, we have conducted a user study to evaluate user performance and learning during a dynamic target acquisition task. The acquisition task was evaluated under a high frame rate, (60 fps) condition, a traditional low frame rate HP condition (10 fps), and the experimental low frame rate LP technique. The task involved the acquisition of targets moving along several different trajectories, modeled after a shotgun trap shooting task. The results of our study indicate the LP condition approaches high frame rate performance within certain classes of target trajectories. Interestingly we also see that learning is consistent across conditions, indicating that it may not always be necessary to train under a visually high frame rate system to learn a particular task. We discuss implications of using the LP technique to mitigate low frame rate issues as well as its potential usefulness for training in low frame rate virtual environments.