Ryan Beams

A significant challenge to the widespread adoption of virtual reality (VR) and augmented reality (AR) environments is cybersickness, which affects a majority of users. With symptoms such as headache, nausea, and blurred vision, cybersickness disrupts the illusion of presence in a virtual environment. Understanding this phenomenon is imperative to democratizing the accessibility and use of this powerful technology. Different visual stimuli induce cybersickness in distinct ways. The psychophysical differences between cybersicknesses induced by distinct sources are largely unknown. Vergence-accommodation conflict (VAC) is believed to contribute to cybersickness. In this paper, we present a systematic user study that specifically analyzes VAC-induced cybersickness. We initiate VAC in a sample of 40 participants by displaying a dynamic sphere that moves in and out of the Zone of Comfort—the region near the focal plane where VAC is minimal. We present evidence for a real-time quantitative measure of VAC-induced cybersickness based on electroencephalogram data. Our findings provide statistically significant evidence that participants exposed to our VAC-inducing condition exhibit decreased beta and gamma spectral power in component clusters with anterior dipole centroids and reduced theta power in clusters with posterior centroids. We present these as candidate correlates rather than a VAC-specific signature because the eyes-open baseline does not match the VAC scene conditions and residual ocular and muscle contributions cannot be fully excluded. We anticipate that this work will lead to the development of strategies to monitor and mitigate VAC-induced cybersickness in virtual environments to improve user experiences and increase adoption of VR and AR.

Marker-based AR image registration is a widely used technique for overlaying patient-specific anatomical models as virtual content using augmented reality head-mounted displays (AR HMDs), providing visual guidance in the real-world environment. The accuracy of AR registration is critical, particularly in surgical applications where sub-millimeter to millimeter precision is optimal. However, there is a lack of standardized evaluation methodologies and metrics for assessing the accuracy and stability of such registrations. To address this gap, we propose a novel evaluation approach for marker-based medical image registration using optical see-through AR HMDs. This method involves direct capture of AR images through the HMD display, enabling objective assessment of both the accuracy and stability of the registered images. Using a precise motion controller, we evaluate AR registrations using image-based and infra-red reflective markers under varying conditions, including different speeds and movements of the AR HMD and the tracked objects, as well as rendering computation loads. The root-mean-square error (RMSE) of the dynamic time warping path matching between the ground truth and the registered image, and latency derived from the cross-correlation of two time-series data are employed to quantify spatial and temporal accuracy. On the HoloLens 2 platform, we demonstrate a method to quantify increased spatial inaccuracy and latency due to the rendering loads and translational speed of the target marker, indicating the deterioration in the accuracy of AR registration. In contrast, during the translational and rotational movement of the AR HMD, the RMSE and the latency remained stable, which is attributed to onboard temporal warping adjustment in rendering.

Real-time image registration and visualization with augmented reality (AR) head-mounted displays (HMDs) are computationally and rendering intensive. Limited computational resources create trade-offs between rendering image resolution, latency, and geometric accuracy. Spatiotemporal image artifacts such as motion blur, multiple images, and flicker span in both spatial and temporal domains when a target or the HMD is in motion. It remains challenging to evaluate the spatiotemporal image quality using conventional methods. In this study, we performed both theoretical simulations and empirical measurements to evaluate the spatiotemporal image visualization on AR image registration. We developed a spatiotemporal visual performance metric combining system imaging characteristics with human visual capabilities. Spatiotemporal image quality was evaluated using various motion and HMD hardware configurations.

We review the literature and relevant standards related to image quality characterization of medical devices using near‐eye displays (NEDs). First, we discuss the experimental considerations for NED measurements and the methods for aligning the light measuring device (LMD) to the NED. Second, we discuss measurement methods for several critical image quality characteristics, namely luminance, contrast, spatial resolution, geometrical distortion, and spatiotemporal properties including the methods' limitations. These characteristics were selected due to the importance of medical device NED applications. Finally, modeling methods for image quality of NEDs are discussed. Accurate and reproducible evaluation methods for image quality are essential for the safe and effective adoption of augmented and virtual reality NEDs into medical applications.

Rapid advancement in virtual reality (VR) technology has broadened its implementation beyond entertainment¹ toward education² and a wide variety of medical applications.^{3 – 5} Clinically, VR has been investigated for the visualization of two-dimensional (2D) and three-dimensional (3D) medical images^6,7 and preoperative surgical planning⁸ in an immersive environment. It has also shown potential for patient-facing vision therapy such as amblyopia treatment^{9 – 11} and pain management.¹² Technically, to create an immersive user experience, VR head-mounted displays (HMDs) present a pair of virtual images on two eyepieces generalizing a stereoscopic visualization of a virtual scene. Unlike on conventional flat-panel displays, the display hardware and optics are independent of two eyepieces. On the other hand, image rendering and processing techniques also interact interocularly on the display pipeline, followed by binocular fusion by the user or the patient. Therefore, visual experience and clinical effectiveness on VR HMDs are affected by both monocular and binocular image quality.

Visual perception on virtual reality head-mounted displays (VR HMDs) involves human vision in the imaging pipeline. Image quality evaluation of VR HMDs may need to be expanded from optical bench testing by incorporating human visual perception. In this study, we implement a 5-degree-of-freedom (5DoF) experimental setup that simulates the human eye geometry and rotation mechanism. Optical modulation transfer function (MTF) measurements are performed using various camera rotation configurations namely pupil rotation, eye rotation, and eye rotation with angle kappa of the human visual system. The measured MTFs of the VR HMD are inserted into a human eye contrast sensitivity model to predict the perceptual contrast sensitivity function (CSF) on a VR HMD. At the same time, we develop a WebXR test platform to perform human observer experiments. Monocular CSFs of human subjects with different interpupillary distance (IPD) are extracted and compared with those calculated from optical MTF measurements. The result shows that image quality, measured as MTF and CSF, degrades at the periphery of display field of view, especially for subjects with an IPD different than that of the HMD. We observed that both the shift of visual point on the HMD eyepiece and the angle between the optical axes of the eye and eyepiece degrade image quality due to optical aberration. The computed CSFs from optical measurement correlates with those of the human observer experiment, with the optimal correlation achieved using the eye rotation with angle kappa setup. The finding demonstrates that more precise image quality assessment can be achieved by integrating eye rotation and human eye contrast sensitivity into optical bench testing.

The measured spatial resolution of augmented reality head‐mounted displays (AR HMDs) is determined by three components: optics, display, and the rendering pipeline, which includes the anti‐aliasing method. Therefore, separating and quantifying the contributions from these components is necessary for evaluating the spatial resolution of AR HMDs. We demonstrate a method for quantifying the contributions from the optical performance and the rendering pipeline on the spatial resolution of AR HMD using point‐spread function (PSF) analysis. In this method, an imaging photometer captures a series of rendered scenes on the AR HMD that consist of arrays of spheres of increasing sizes with different anti‐aliasing (AA) methods. The contributions from the optics, display, and anti‐aliasing methods can be separated, and we find that temporal anti‐aliasing (TAA) and multisample anti‐aliasing (MSAA) are in good agreement with the performance of no AA. However, fast approximate anti‐aliasing (FXAA) results in decreased luminance for smaller rendered targets and a different spatial resolution for larger targets. Finally, we repeated our methods on multiple HMDs and characterized the dependence of the spatial resolution across the field of view. These measurements show significant non‐uniformity in the optical contribution to the spatial resolution.

Video see‐through (VST) head‐mounted displays (HMDs) reconstruct the physical world using outward‐facing cameras and depth sensors. One of the major challenges for VST HMDs is the geometric distortion of the physical pass‐through content. In this study, we present the experimental setup and metrology for VST geometric distortion measurement on single‐camera and dual‐camera VST HMDs. The VST geometric distortion is compared to the digital distortion by the HMD. Finally, we investigate the impacts of camera misalignment and depth distance on VST geometric distortion without and with angular‐based distortion correction. The results demonstrate that camera misalignment with respect to the eye position is the primary root cause for the distortion. Dual‐camera setup is necessary to minimize the VST geometric distortion by placing the cameras close to the eye positions. Although distortion correction can partially mitigate VST geometric distortion in a 2D plane, it remains a challenge for VST distortion correction at multiple depth distance in the 3D space. The method and findings presented of this study may provide guidelines for evaluation of spatial accuracy on VST HMDs and hardware design.

The potential for extended reality devices in medicine has recently grown. In order to approve these devices for medical use, they must be evaluated on many characteristics to ensure safety and effectiveness. One of the most important evaluations is the quality of the device's display, known as image quality, especially for medical applications (Zhao et al., 2023). The image quality evaluation of virtual and augmented reality devices requires the creation of a variety of different test patterns. These test patterns consist of many different shapes that must frequently change their location, size, or color in order to adapt to the many headsets in the market (Beams et al., 2020) (Zhao et al., 2023). The Pattern Creation Tool allows for the creation, editing, and sharing of patterns that work in a mixed reality environment. The Pattern Creation Tool includes multiple default patterns that are commonly used in image quality evaluation. These include a pattern for finding the center of the device's display, a pattern to measure geometric distortion, a pattern to measure Michelson contrast, and a pattern to measure line spread. These patterns can make simple testing easier as well as serve as a template for other patterns.