Alvaro Uribe-Quevedo

Advances in consumer virtual reality (VR) and artificial intelligence (AI) have accelerated the use of immersive virtual learning environments (iVLEs) for skills training. Learner engagement is a critical determinant of training effectiveness, which can be shaped by VR system features (e.g., visual, auditory, and tactile immersion) coupled with interaction mechanics and instructional design integrated with the instructional behaviors of virtual human assistants (VHAs). Although visual and behavioral fidelity in VHAs have been extensively studied, functional fidelity (i.e., the extent to which the iVLE and/or VHAs support cognitive, perceptual, and motor processes required to perform a task regardless of visual realism), and particularly the temporal alignment of instructional guidance with learners’ cognitive and motor demands, remains underexamined. This article highlights research on VHAs in iVLEs with a special emphasis on temporal functional fidelity as an emerging requirement for synchronizing instructional support with user workload and task phases. By consolidating existing findings and highlighting gaps in current empirical work, this article outlines key implications for the design and evaluation of VHAs and identifies directions for future research aimed at optimizing instructional timing in iVLEs. The goal is to inform principled VHA design and clarify how fidelity dimensions should be integrated to support effective, pedagogically grounded immersive learning experiences.

Driving in snowy conditions challenges both human drivers and autonomous systems. Snowfall and ice accumulation impair vehicle control and affect driver perception and performance. Road markings are often obscured, forcing drivers to rely on intuition and memory to stay in their lane, which can lead to encroachment into adjacent lanes or sidewalks. Current lane detectors assist in lane keeping, but their performance is compromised by visual disturbances such as ice reflection, snowflake movement, fog, and snow cover. Furthermore, testing these systems with users on actual snowy roads involves risks to driver safety, equipment integrity, and ethical compliance. This study presents a low-cost virtual reality simulation for evaluating winter lane detection in controlled and safe conditions from a human-in-the-loop perspective. Participants drove in a simulated snowy scenario with and without the detector while quantitative and qualitative variables were monitored. Results showed a 49.9% reduction in unintentional lane departures with the detector and significantly improved user experience, as measured by the UEQ-S (p = 0.023, Cohen’s d = 0.72). Participants also reported higher perceived safety, situational awareness, and confidence. These findings highlight the potential of vision-based lane detection systems adapted to winter environments and demonstrate the value of immersive simulations for user-centered testing of ADASs.

Artificial Intelligence (AI), computer simulations, and virtual reality (VR) are increasingly becoming accessible tools that can be leveraged to implement training protocols and educational resources. Typical assessment tools related to sensory and neural processing associated with task performance in virtual environments often rely on self-reported surveys, unlike electroencephalography (EEG), which is often used to compare the effects of different types of sensory feedback (e.g., auditory, visual, and haptic) in simulation environments in an objective manner. However, it can be challenging to know which aspects of the EEG signal represent the impact of different types of sensory feedback on neural processing. Machine learning approaches offer a promising direction for identifying EEG signal features that differentiate the impact of different types of sensory feedback during simulation training. For the current study, machine learning techniques were applied to differentiate neural circuitry associated with haptic and non-haptic feedback in a simulated drilling task. Nine EEG channels were selected and analyzed, extracting different time-domain, frequency-domain, and nonlinear features, where 360 features were tested (40 features per channel). A feature selection stage identified the most relevant features, including the Hurst exponent of 13–21 Hz, kurtosis of 21–30 Hz, power spectral density of 21–30 Hz, variance of 21–30 Hz, and spectral entropy of 13–21 Hz. Using those five features, trials with haptic feedback were correctly identified from those without haptic feedback with an accuracy exceeding 90%, increasing to 99% when using 10 features. These results show promise for the future application of machine learning approaches to predict the impact of haptic feedback on neural processing during VR protocols involving drilling tasks, which can inform future applications of VR and simulation for occupational skill acquisition.

Virtual reality (VR) enables the development of virtual training frameworks suitable for various domains, especially when real-world conditions may be hazardous or impossible to replicate because of unique additional resources (e.g., equipment, infrastructure, people, locations). Although VR technology has significantly advanced in recent years, methods for evaluating immersion (i.e., the extent to which the user is engaged with the sensory information from the virtual environment or is invested in the intended task) continue to rely on self-reported questionnaires, which are often administered after using the virtual scenario. Having an objective method to measure immersion is particularly important when using VR for training, education, and applications that promote the development, fine-tuning, or maintenance of skills. The level of immersion may impact performance and the translation of knowledge and skills to the real-world. This is particularly important in tasks where motor skills are combined with complex decision making, such as surgical procedures. Efforts to better measure immersion have included the use of physiological measurements including heart rate and skin response, but so far they do not offer robust metrics that provide the sensitivity to discriminate different states (idle, easy, and hard), which is critical when using VR for training to determine how successful the training is in engaging the user’s senses and challenging their cognitive capabilities. In this study, electroencephalography (EEG) data were collected from 14 participants who completed VR jigsaw puzzles with two different levels of task difficulty. Machine learning was able to accurately classify the EEG data collected during three different states, obtaining accuracy rates of 86% and 97% for differentiating easy versus hard difficulty states and baseline vs. VR states. Building on these results may enable the identification of robust biomarkers of immersion in VR, enabling real-time recognition of the level of immersion that can be used to design more effective and translative VR-based training. This method has the potential to adjust aspects of VR related to task difficulty to ensure that participants are immersed in VR.

Virtual reality (VR) is gaining popularity as an educational, training, and healthcare tool due to its decreasing cost. Because of the high user variability in terms of ergonomics, 3D manipulation techniques (3DMTs) for 3D user interfaces (3DUIs) must be adjustable for comfort and usability, hence avoiding interactions that only function for the typical user. Given the role of the upper limb (i.e., arm, forearm, and hands) in interacting with virtual objects, research has led to the development of 3DMTs for facilitating isomorphic (i.e., an equal translation of controller movement) and non-isomorphic (i.e., adjusted controller visuals in VR) interactions. Although advances in 3DMTs have been proven to facilitate VR interactions, user variability has not been addressed in terms of ergonomics. This work introduces Piecewise, an upper-limb-customized non-isomorphic 3DMT for 3DUIs that accounts for user variability by incorporating upper-limb ergonomics and comfort range of motion. Our research investigates the effects of upper-limb ergonomics on time completion, skipped objects, percentage of reach, upper-body lean, engagement, and presence levels in comparison to common 3DMTs, such as normal (physical reach), object translation, and reach-bounded non-linear input amplification (RBNLIA). A 20-person within-subjects study revealed that upper-limb ergonomics influence the execution and perception of tasks in virtual reality. The proposed Piecewise approach ranked second behind the RBNLIA method, although all 3DMTs were evaluated as usable, engaging, and favorable in general. The implications of our research are significant because upper-limb ergonomics can affect VR performance for a broader range of users as the technology becomes widely available and adopted for accessibility and inclusive design, providing opportunities to provide additional customizations that can affect the VR user experience.