GWGuande Wu

Guande Wu

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Guande Wu has a total of 45 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    InsightAR: A Tool for Multi-modal Summarization and Interactive Analysis of AR-based Egocentric Task Videos

    by Guande Wu, Dishita Turakhia, Eden Wu, Sonia Castelo Quispe, João Rulff, Erin McGowan, Jianben He, Yawei Wang, Jing Qian, Claudio Silva

    The growing use of head-mounted AR devices has produced a surge of egocentric recordings of procedural tasks—step-by-step activities such as medical procedures—creating rich opportunities for performance analysis. Yet current analyses are time-consuming, requiring substantial manual inspection to identify events, actions, and objects from videos, with little automated support for surfacing insights or errors. We introduce InsightAR, a visual analytics tool that integrates event detection, action recognition, and multi-modal language models to create video summaries capturing key errors and insights. Its interface further enables users to refine generated insights and errors collaboratively. A user study (N=16) found that InsightAR identifies significantly more errors and marginally significant insight numbers than the baseline. Our work demonstrated the feasibility and potential benefits of future human-AI collaborative video analysis.

  • Top Cited Papers
    Open Access

    Satori 悟り: Towards Proactive AR Assistant with Belief-Desire-Intention User Modeling

    by Chenyi Li, Guande Wu, Gromit Yeuk-Yin Chan, Dishita Gdi Turakhia, Sonia Castelo Quispe, Dong Li, Leslie Welch, Claudio Silva, Jing Qian


  • Design and Implementation of the Transparent, Interpretable, and Multimodal (TIM) AR Personal Assistant

    by Erin McGowan, Joao Rulff, Sonia Castelo, Guande Wu, Shaoyu Chen, Roque Lopez, Bea Steers, Iran R. Roman, Fábio F. Dias, Jing Qian, et al.

    The concept of an AI assistant for task guidance is rapidly shifting from a science fiction staple to an impending reality. Such a system is inherently complex, requiring models for perceptual grounding, attention, and reasoning, an intuitive interface that adapts to the performer's needs, and the orchestration of data streams from many sensors. Moreover, all data acquired by the system must be readily available for post-hoc analysis to enable developers to understand performer behavior and quickly detect failures. We introduce TIM, the first end-to-end AI-enabled task guidance system in augmented reality which is capable of detecting both the user and scene as well as providing adaptable, just-in-time feedback. We discuss the system challenges and propose design solutions. We also demonstrate how TIM adapts to domain applications with varying needs, highlighting how the system components can be customized for each scenario.


  • Open Access

    Exploring the "Freedom to be Me" through Design Sprints with Neurodivergent Scholars

    by LouAnne Boyd, Annuska Zolyomi, Saad Hassan, Seray B Ibrahim, Guande Wu, Kay Kender

    Social stigma negatively impacts the well-being of neurodivergent individuals. Specifically for autistic people, the social isolation and pressure to conform to normative ways of being can take a tremendous toll; such as a thwarted sense of belonging to the point of higher rates of suicide. Yet, few technologies are directly targeting the problem of stigma. Socio-technical systems have tremendous potential to shift public perception of traditionally marginalized populations. However, these systems are not consistently designed to reflect the values and needs of neurodivergent individuals. This work explores the use of design sprints to envision CT– a spectrum of technology that could reduce social stigma by increasing the public’s awareness, accommodations, acceptance, advocacy, and appreciation. This work reports on two design sprints across 25 HCI community members with varying lived experiences with neurodiversity and knowledge of design practice. The resulting design concepts were discussed in the groups and then analyzed to reflect on how they might combat stigma. Results reveal designs that support the freedom to be oneself via (1) safe spaces (2) public understanding, and (3) authentic expression of strengths, challenges, and needs.


  • HuBar: A Visual Analytics Tool to Explore Human Behavior Based on fNIRS in AR Guidance Systems

    by Sonia Castelo, Joao Rulff, Parikshit Solunke, Erin McGowan, Guande Wu, Iran Roman, Roque Lopez, Bea Steers, Qi Sun, Juan Bello, et al.

    The concept of an intelligent augmented reality (AR) assistant has significant, wide-ranging applications, with potential uses in medicine, military, and mechanics domains. Such an assistant must be able to perceive the environment and actions, reason about the environment state in relation to a given task, and seamlessly interact with the task performer. These interactions typically involve an AR headset equipped with sensors which capture video, audio, and haptic feedback. Previous works have sought to facilitate the development of intelligent AR assistants by visualizing these sensor data streams in conjunction with the assistant's perception and reasoning model outputs. However, existing visual analytics systems do not focus on user modeling or include biometric data, and are only capable of visualizing a single task session for a single performer at a time. Moreover, they typically assume a task involves linear progression from one step to the next. We propose a visual analytics system that allows users to compare performance during multiple task sessions, focusing on non-linear tasks where different step sequences can lead to success. In particular, we design visualizations for understanding user behavior through functional near-infrared spectroscopy (fNIRS) data as a proxy for perception, attention, and memory as well as corresponding motion data (acceleration, angular velocity, and gaze). We distill these insights into embedding representations that allow users to easily select groups of sessions with similar behaviors. We provide two case studies that demonstrate how to use these visualizations to gain insights about task performance using data collected during helicopter copilot training tasks. Finally, we evaluate our approach through an in-depth examination of a think-aloud experiment with five domain experts.


  • Open Access

    HuBar: A Visual Analytics Tool to Explore Human Behaviour based on fNIRS in AR guidance systems

    by Sonia Castelo, Joao Rulff, Parikshit Solunke, Erin McGowan, Guande Wu, Iran Roman, Roque Lopez, Bea Steers, Qi Sun, Juan Bello, et al.

    The concept of an intelligent augmented reality (AR) assistant has significant, wide-ranging applications, with potential uses in medicine, military, and mechanics domains. Such an assistant must be able to perceive the environment and actions, reason about the environment state in relation to a given task, and seamlessly interact with the task performer. These interactions typically involve an AR headset equipped with sensors which capture video, audio, and haptic feedback. Previous works have sought to facilitate the development of intelligent AR assistants by visualizing these sensor data streams in conjunction with the assistant's perception and reasoning model outputs. However, existing visual analytics systems do not focus on user modeling or include biometric data, and are only capable of visualizing a single task session for a single performer at a time. Moreover, they typically assume a task involves linear progression from one step to the next. We propose a visual analytics system that allows users to compare performance during multiple task sessions, focusing on non-linear tasks where different step sequences can lead to success. In particular, we design visualizations for understanding user behavior through functional near-infrared spectroscopy (fNIRS) data as a proxy for perception, attention, and memory as well as corresponding motion data (acceleration, angular velocity, and gaze). We distill these insights into embedding representations that allow users to easily select groups of sessions with similar behaviors. We provide two case studies that demonstrate how to use these visualizations to gain insights about task performance using data collected during helicopter copilot training tasks. Finally, we evaluate our approach by conducting an in-depth examination of a think-aloud experiment with five domain experts.

  • Open Access

    ARTiST: Automated Text Simplification for Task Guidance in Augmented Reality

    by Guande Wu, Jing Qian, Sonia Castelo Quispe, Shaoyu Chen, João Rulff, Claudio Silva

    Text presented in augmented reality provides in-situ, real-time information for users. However, this content can be challenging to apprehend quickly when engaging in cognitively demanding AR tasks, especially when it is presented on a head-mounted display. We propose ARTiST, an automatic text simplification system that uses a few-shot prompt and GPT-3 models to specifically optimize the text length and semantic content for augmented reality. Developed out of a formative study that included seven users and three experts, our system combines a customized error calibration model with a few-shot prompt to integrate the syntactic, lexical, elaborative, and content simplification techniques, and generate simplified AR text for head-worn displays. Results from a 16-user empirical study showed that ARTiST lightens the cognitive load and improves performance significantly over both unmodified text and text modified via traditional methods. Our work constitutes a step towards automating the optimization of batch text data for readability and performance in augmented reality.


  • Socrates: Data Story Generation via Adaptive Machine-Guided Elicitation of User Feedback

    by Guande Wu, Shunan Guo, Jane Hoffswell, Gromit Yeuk-Yin Chan, Ryan A. Rossi, Eunyee Koh

    Visual data stories can effectively convey insights from data, yet their creation often necessitates intricate data exploration, insight discovery, narrative organization, and customization to meet the communication objectives of the storyteller. Existing automated data storytelling techniques, however, tend to overlook the importance of user customization during the data story authoring process, limiting the system's ability to create tailored narratives that reflect the user's intentions. We present a novel data story generation workflow that leverages adaptive machine-guided elicitation of user feedback to customize the story. Our approach employs an adaptive plug-in module for existing story generation systems, which incorporates user feedback through interactive questioning based on the conversation history and dataset. This adaptability refines the system's understanding of the user's intentions, ensuring the final narrative aligns with their goals. We demonstrate the feasibility of our approach through the implementation of an interactive prototype: Socrates. Through a quantitative user study with 18 participants that compares our method to a state-of-the-art data story generation algorithm, we show that Socrates produces more relevant stories with a larger overlap of insights compared to human-generated stories. We also demonstrate the usability of Socrates via interviews with three data analysts and highlight areas of future work.


  • ARGUS: Visualization of AI-Assisted Task Guidance in AR

    by Sonia Castelo, Joao Rulff, Erin McGowan, Bea Steers, Guande Wu, Shaoyu Chen, Iran Roman, Roque Lopez, Ethan Brewer, Chen Zhao, et al.

    The concept of augmented reality (AR) assistants has captured the human imagination for decades, becoming a staple of modern science fiction. To pursue this goal, it is necessary to develop artificial intelligence (AI)-based methods that simultaneously perceive the 3D environment, reason about physical tasks, and model the performer, all in real-time. Within this framework, a wide variety of sensors are needed to generate data across different modalities, such as audio, video, depth, speech, and time-of-flight. The required sensors are typically part of the AR headset, providing performer sensing and interaction through visual, audio, and haptic feedback. AI assistants not only record the performer as they perform activities, but also require machine learning (ML) models to understand and assist the performer as they interact with the physical world. Therefore, developing such assistants is a challenging task. We propose ARGUS, a visual analytics system to support the development of intelligent AR assistants. Our system was designed as part of a multi-year-long collaboration between visualization researchers and ML and AR experts. This co-design process has led to advances in the visualization of ML in AR. Our system allows for online visualization of object, action, and step detection as well as offline analysis of previously recorded AR sessions. It visualizes not only the multimodal sensor data streams but also the output of the ML models. This allows developers to gain insights into the performer activities as well as the ML models, helping them troubleshoot, improve, and fine-tune the components of the AR assistant.