Kevin Ponto

While virtual reality (VR) is often thought of as a new technology, the foundations for creating artificial immersive environments have a long, rich history

Obstacle negotiation during locomotion depends on the integration of exteroceptive information about the environment, proprioceptive signals from the body, and exproprioceptive visual feedback about the limbs. This study examined how removing visual limb information and introducing VR-specific sensory uncertainty affect overground obstacle-crossing behavior. Participants walked under three conditions: natural environment with full vision, natural environment with lower-limb occlusion, and immersive VR without a lower-limb representation. Removing limb vision in the real world selectively increased toe clearance while leaving baseline gait unchanged, demonstrating the role of exproprioceptive feedback in fine-tuning foot trajectory. VR amplified these adaptations, yielding slower speeds, wider bases of support, and even greater clearance margins, reflecting compounded uncertainty from altered exteroceptive cues. Yet obstacle location effects were consistent across environments, suggesting preserved underlying control and the potential for a scaling relationship between VR and real-world performance. Findings highlight key design considerations for VR-based gait assessment and rehabilitation.

First responders operate in high-stakes environments, demanding rapid, accurate decision-making. Recent research has provided recommendations on how Augmented Reality (AR) could be utilized to support their efforts. Building off of these studies, this paper presents EasyVizAR, an AR system designed to enhance situational awareness and operational efficiency in challenging indoor scenarios. Leveraging edge computing and advanced computer vision techniques, EasyVizAR addresses critical challenges, such as object detection, localization, and information sharing. This research details the system’s architecture, including its use of ParaDrop-based edge computing, and explores its application in rescue and active-shooter scenarios. We present our work in developing key features, including real-time object saliency cues, improved object detection, person identification, multi-user 3D map generation and visualization, and multimodal AR navigation cues.

Observing users interacting with a learning game, often referred to as playtesting, is a critical component of usability testing. Unfortunately, this practice is expensive, requiring users and researchers to be in the same place at the same time as the participants. With Virtual Reality, this difficulty is amplified due to the experience being hidden from researcher view by default, and the extra complexity of setting up casting to an external monitor, especially with groups. In this paper we develop and utilize a replay-based approach to usability testing that relieves these concerns. The approach uses a low-bandwidth stream of telemetry signals that are generated by the original play session. These signals are then reconstructed into a full representation of the original experience at a different time or place and leveraged to identify usability issues. Once the issues have been discovered, automated processes are developed to computationally identify their presence and severity in arbitrarily large public audiences. This work contributes a demonstrated use of replay for Virtual Reality usability research, and a novel use of replay combined with educational data mining to develop an automated process for studying large audiences at low cost.

First responders (FRs) navigate hazardous, unfamiliar environments in the field (e.g., mass-casualty incidents), making life-changing decisions in a split second. AR head-mounted displays (HMDs) have shown promise in supporting them due to its capability of recognizing and augmenting the challenging environments in a hands-free manner. However, the design space have not been thoroughly explored by involving various FRs who serve different roles (e.g., firefighters, law enforcement) but collaborate closely in the field. We interviewed 26 first responders in the field who experienced a state-of-the-art optical-see-through AR HMD, as well as its interaction techniques and four types of AR cues (i.e., overview cues, directional cues, highlighting cues, and labeling cues), soliciting their first-hand experiences, design ideas, and concerns. Our study revealed both generic and role-specific preferences and needs for AR hardware, interactions, and feedback, as well as identifying desired AR designs tailored to urgent, risky scenarios (e.g., affordance augmentation to facilitate fast and safe action). While acknowledging the value of AR HMDs, concerns were also raised around trust, privacy, and proper integration with other equipment. Finally, we derived comprehensive and actionable design guidelines to inform future AR systems for in-field FRs.

Background: Home assessment is a critical component of successful home modifications, enabling individuals with functional limitations to age in place comfortably. A high-quality home assessment tool should facilitate a valid and reliable assessment involving health care and housing professionals, while also engaging and empowering consumers and their caregivers who may be dealing with multiple functional limitations. Unlike traditional paper-and-pencil assessments, which require extensive training and expert knowledge and can be alienating to consumers, mobile health (mHealth) apps have the potential to engage all parties involved, empowering and activating consumers to take action. However, little is known about which apps contain all the necessary functionality, quality appraisal, and accessibility. Objective: This study aimed to assess the functionality, overall quality, and accessibility of mHealth home assessment apps. Methods: mHealth apps enabling home assessment for aging in place were identified through a comprehensive search of scholarly articles, the Apple (iOS) and Google Play (Android) stores in the United States, and fnd.io. The search was conducted between November 2022 and January 2023 following a method adapted from PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). Reviewers performed a content analysis of the mobile app features to evaluate their functionality, overall quality, and accessibility. The functionality assessment used a home assessment component matrix specifically developed for this study. For overall quality, the Mobile Application Rating Scale (MARS) was used to determine the apps’ effectiveness in engaging and activating consumers and their caregivers. Accessibility was assessed using the Web Content Accessibility Guidelines (WCAG) 2.1 (A and AA levels). These 3 assessments were synthesized and visualized to provide a comprehensive evaluation. Results: A total of 698 apps were initially identified. After further screening, only 6 apps remained. Our review revealed that none of the apps used thoroughly tested assessment tools, offered all the functionality required for reliable home assessment, achieved the “good” quality threshold as measured by the MARS, or met the accessibility criteria when evaluated against WCAG 2.1. However, DIYModify received the highest scores in both the overall quality and accessibility assessments. The MapIt apps also showed significant potential due to their ability to measure the 3D environment and the inclusion of a desktop version that extends the app’s functionality. Conclusions: Our review revealed that there are very few apps available within the United States that possess the necessary functionality, engaging qualities, and accessibility to effectively activate consumers and their caregivers for successful home modification. Future app development should prioritize the integration of reliable and thoroughly tested assessment tools as the foundation of the development process. Furthermore, efforts should be made to enhance the overall quality and accessibility of these apps to better engage and empower consumers to take necessary actions to age in place.

In means of improving the accessibility of performing home assessments, we have developed a novel set of augmented reality (AR) tools. To test the impact of these tools, a user study was run that compares the developed tools against traditional techniques. The findings show that the AR assessment tools generally provide results that are both more accurate and quicker to obtain compared to traditional methods. In addition, participants rated the AR assessment tools as easier to use and were more confident in their assessments compared to when using traditional measures. These results demonstrate that current generation phone-based AR technologies can be utilized as a powerful tool in the assessment process.