Guande Wu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Guande Wu has a total of 45 co-authors. Here are the top co-authors by number of shared publications:
- Jing Qian
New York University, New York, New York, USA
7 shared publications - Joao Rulff
New York University, New York, NY, USA
7 shared publications - Sonia Castelo
New York University, New York, New York, USA
6 shared publications - Bea Steers
New York University, New York, NY, USA
5 shared publications - Claudio Silva
New York University
5 shared publications - Erin McGowan
New York University, New York, New York, USA
5 shared publications
Latest Publications
- Open Access
- 4 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 141-150
CHORUS: Designing Human–AI Multi-Agent Collaboration for Professional Translators
- 4 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 141-150
Open AccessCHORUS: Designing Human–AI Multi-Agent Collaboration for Professional Translators
- Open Access
- 7 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-9
InsightAR: A Tool for Multi-modal Summarization and Interactive Analysis of AR-based Egocentric Task Videos
The growing use of head-mounted AR devices has produced a surge of egocentric recordings of procedural tasks—step-by-step activities such as medical procedures—creating rich opportunities for performance analysis. Yet current analyses are time-consuming, requiring substantial manual inspection to identify events, actions, and objects from videos, with little automated support for surfacing insights or errors. We introduce InsightAR, a visual analytics tool that integrates event detection, action recognition, and multi-modal language models to create video summaries capturing key errors and insights. Its interface further enables users to refine generated insights and errors collaboratively. A user study (N=16) found that InsightAR identifies significantly more errors and marginally significant insight numbers than the baseline. Our work demonstrated the feasibility and potential benefits of future human-AI collaborative video analysis.
- 7 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-9
Open AccessInsightAR: A Tool for Multi-modal Summarization and Interactive Analysis of AR-based Egocentric Task Videos
The growing use of head-mounted AR devices has produced a surge of egocentric recordings of procedural tasks—step-by-step activities such as medical procedures—creating rich opportunities for performance analysis. Yet current analyses are time-consuming, requiring substantial manual inspection to identify events, actions, and objects from videos, with little automated support for surfacing insights or errors. We introduce InsightAR, a visual analytics tool that integrates event detection, action recognition, and multi-modal language models to create video summaries capturing key errors and insights. Its interface further enables users to refine generated insights and errors collaboratively. A user study (N=16) found that InsightAR identifies significantly more errors and marginally significant insight numbers than the baseline. Our work demonstrated the feasibility and potential benefits of future human-AI collaborative video analysis.
- Top Cited PapersOpen Access
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-24
Satori 悟り: Towards Proactive AR Assistant with Belief-Desire-Intention User Modeling
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-24
Top Cited PapersOpen AccessSatori 悟り: Towards Proactive AR Assistant with Belief-Desire-Intention User Modeling
- 10 March 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Graphics and Applications
- Vol. 45 (1) , 28-42
Design and Implementation of the Transparent, Interpretable, and Multimodal (TIM) AR Personal Assistant
The concept of an AI assistant for task guidance is rapidly shifting from a science fiction staple to an impending reality. Such a system is inherently complex, requiring models for perceptual grounding, attention, and reasoning, an intuitive interface that adapts to the performer's needs, and the orchestration of data streams from many sensors. Moreover, all data acquired by the system must be readily available for post-hoc analysis to enable developers to understand performer behavior and quickly detect failures. We introduce TIM, the first end-to-end AI-enabled task guidance system in augmented reality which is capable of detecting both the user and scene as well as providing adaptable, just-in-time feedback. We discuss the system challenges and propose design solutions. We also demonstrate how TIM adapts to domain applications with varying needs, highlighting how the system components can be customized for each scenario.
- 10 March 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Graphics and Applications
- Vol. 45 (1) , 28-42
Design and Implementation of the Transparent, Interpretable, and Multimodal (TIM) AR Personal Assistant
The concept of an AI assistant for task guidance is rapidly shifting from a science fiction staple to an impending reality. Such a system is inherently complex, requiring models for perceptual grounding, attention, and reasoning, an intuitive interface that adapts to the performer's needs, and the orchestration of data streams from many sensors. Moreover, all data acquired by the system must be readily available for post-hoc analysis to enable developers to understand performer behavior and quickly detect failures. We introduce TIM, the first end-to-end AI-enabled task guidance system in augmented reality which is capable of detecting both the user and scene as well as providing adaptable, just-in-time feedback. We discuss the system challenges and propose design solutions. We also demonstrate how TIM adapts to domain applications with varying needs, highlighting how the system components can be customized for each scenario.
- Open Access
- 27 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Exploring the "Freedom to be Me" through Design Sprints with Neurodivergent Scholars
Social stigma negatively impacts the well-being of neurodivergent individuals. Specifically for autistic people, the social isolation and pressure to conform to normative ways of being can take a tremendous toll; such as a thwarted sense of belonging to the point of higher rates of suicide. Yet, few technologies are directly targeting the problem of stigma. Socio-technical systems have tremendous potential to shift public perception of traditionally marginalized populations. However, these systems are not consistently designed to reflect the values and needs of neurodivergent individuals. This work explores the use of design sprints to envision CT– a spectrum of technology that could reduce social stigma by increasing the public’s awareness, accommodations, acceptance, advocacy, and appreciation. This work reports on two design sprints across 25 HCI community members with varying lived experiences with neurodiversity and knowledge of design practice. The resulting design concepts were discussed in the groups and then analyzed to reflect on how they might combat stigma. Results reveal designs that support the freedom to be oneself via (1) safe spaces (2) public understanding, and (3) authentic expression of strengths, challenges, and needs.
- 27 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Open AccessExploring the "Freedom to be Me" through Design Sprints with Neurodivergent Scholars
Social stigma negatively impacts the well-being of neurodivergent individuals. Specifically for autistic people, the social isolation and pressure to conform to normative ways of being can take a tremendous toll; such as a thwarted sense of belonging to the point of higher rates of suicide. Yet, few technologies are directly targeting the problem of stigma. Socio-technical systems have tremendous potential to shift public perception of traditionally marginalized populations. However, these systems are not consistently designed to reflect the values and needs of neurodivergent individuals. This work explores the use of design sprints to envision CT– a spectrum of technology that could reduce social stigma by increasing the public’s awareness, accommodations, acceptance, advocacy, and appreciation. This work reports on two design sprints across 25 HCI community members with varying lived experiences with neurodiversity and knowledge of design practice. The resulting design concepts were discussed in the groups and then analyzed to reflect on how they might combat stigma. Results reveal designs that support the freedom to be oneself via (1) safe spaces (2) public understanding, and (3) authentic expression of strengths, challenges, and needs.
- 9 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 31 (1) , 119-129
HuBar: A Visual Analytics Tool to Explore Human Behavior Based on fNIRS in AR Guidance Systems
The concept of an intelligent augmented reality (AR) assistant has significant, wide-ranging applications, with potential uses in medicine, military, and mechanics domains. Such an assistant must be able to perceive the environment and actions, reason about the environment state in relation to a given task, and seamlessly interact with the task performer. These interactions typically involve an AR headset equipped with sensors which capture video, audio, and haptic feedback. Previous works have sought to facilitate the development of intelligent AR assistants by visualizing these sensor data streams in conjunction with the assistant's perception and reasoning model outputs. However, existing visual analytics systems do not focus on user modeling or include biometric data, and are only capable of visualizing a single task session for a single performer at a time. Moreover, they typically assume a task involves linear progression from one step to the next. We propose a visual analytics system that allows users to compare performance during multiple task sessions, focusing on non-linear tasks where different step sequences can lead to success. In particular, we design visualizations for understanding user behavior through functional near-infrared spectroscopy (fNIRS) data as a proxy for perception, attention, and memory as well as corresponding motion data (acceleration, angular velocity, and gaze). We distill these insights into embedding representations that allow users to easily select groups of sessions with similar behaviors. We provide two case studies that demonstrate how to use these visualizations to gain insights about task performance using data collected during helicopter copilot training tasks. Finally, we evaluate our approach through an in-depth examination of a think-aloud experiment with five domain experts.
- 9 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 31 (1) , 119-129
HuBar: A Visual Analytics Tool to Explore Human Behavior Based on fNIRS in AR Guidance Systems
The concept of an intelligent augmented reality (AR) assistant has significant, wide-ranging applications, with potential uses in medicine, military, and mechanics domains. Such an assistant must be able to perceive the environment and actions, reason about the environment state in relation to a given task, and seamlessly interact with the task performer. These interactions typically involve an AR headset equipped with sensors which capture video, audio, and haptic feedback. Previous works have sought to facilitate the development of intelligent AR assistants by visualizing these sensor data streams in conjunction with the assistant's perception and reasoning model outputs. However, existing visual analytics systems do not focus on user modeling or include biometric data, and are only capable of visualizing a single task session for a single performer at a time. Moreover, they typically assume a task involves linear progression from one step to the next. We propose a visual analytics system that allows users to compare performance during multiple task sessions, focusing on non-linear tasks where different step sequences can lead to success. In particular, we design visualizations for understanding user behavior through functional near-infrared spectroscopy (fNIRS) data as a proxy for perception, attention, and memory as well as corresponding motion data (acceleration, angular velocity, and gaze). We distill these insights into embedding representations that allow users to easily select groups of sessions with similar behaviors. We provide two case studies that demonstrate how to use these visualizations to gain insights about task performance using data collected during helicopter copilot training tasks. Finally, we evaluate our approach through an in-depth examination of a think-aloud experiment with five domain experts.
- Open Access
HuBar: A Visual Analytics Tool to Explore Human Behaviour based on fNIRS in AR guidance systems
The concept of an intelligent augmented reality (AR) assistant has significant, wide-ranging applications, with potential uses in medicine, military, and mechanics domains. Such an assistant must be able to perceive the environment and actions, reason about the environment state in relation to a given task, and seamlessly interact with the task performer. These interactions typically involve an AR headset equipped with sensors which capture video, audio, and haptic feedback. Previous works have sought to facilitate the development of intelligent AR assistants by visualizing these sensor data streams in conjunction with the assistant's perception and reasoning model outputs. However, existing visual analytics systems do not focus on user modeling or include biometric data, and are only capable of visualizing a single task session for a single performer at a time. Moreover, they typically assume a task involves linear progression from one step to the next. We propose a visual analytics system that allows users to compare performance during multiple task sessions, focusing on non-linear tasks where different step sequences can lead to success. In particular, we design visualizations for understanding user behavior through functional near-infrared spectroscopy (fNIRS) data as a proxy for perception, attention, and memory as well as corresponding motion data (acceleration, angular velocity, and gaze). We distill these insights into embedding representations that allow users to easily select groups of sessions with similar behaviors. We provide two case studies that demonstrate how to use these visualizations to gain insights about task performance using data collected during helicopter copilot training tasks. Finally, we evaluate our approach by conducting an in-depth examination of a think-aloud experiment with five domain experts.
Open AccessHuBar: A Visual Analytics Tool to Explore Human Behaviour based on fNIRS in AR guidance systems
The concept of an intelligent augmented reality (AR) assistant has significant, wide-ranging applications, with potential uses in medicine, military, and mechanics domains. Such an assistant must be able to perceive the environment and actions, reason about the environment state in relation to a given task, and seamlessly interact with the task performer. These interactions typically involve an AR headset equipped with sensors which capture video, audio, and haptic feedback. Previous works have sought to facilitate the development of intelligent AR assistants by visualizing these sensor data streams in conjunction with the assistant's perception and reasoning model outputs. However, existing visual analytics systems do not focus on user modeling or include biometric data, and are only capable of visualizing a single task session for a single performer at a time. Moreover, they typically assume a task involves linear progression from one step to the next. We propose a visual analytics system that allows users to compare performance during multiple task sessions, focusing on non-linear tasks where different step sequences can lead to success. In particular, we design visualizations for understanding user behavior through functional near-infrared spectroscopy (fNIRS) data as a proxy for perception, attention, and memory as well as corresponding motion data (acceleration, angular velocity, and gaze). We distill these insights into embedding representations that allow users to easily select groups of sessions with similar behaviors. We provide two case studies that demonstrate how to use these visualizations to gain insights about task performance using data collected during helicopter copilot training tasks. Finally, we evaluate our approach by conducting an in-depth examination of a think-aloud experiment with five domain experts.
- Open Access
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-24
ARTiST: Automated Text Simplification for Task Guidance in Augmented Reality
Text presented in augmented reality provides in-situ, real-time information for users. However, this content can be challenging to apprehend quickly when engaging in cognitively demanding AR tasks, especially when it is presented on a head-mounted display. We propose ARTiST, an automatic text simplification system that uses a few-shot prompt and GPT-3 models to specifically optimize the text length and semantic content for augmented reality. Developed out of a formative study that included seven users and three experts, our system combines a customized error calibration model with a few-shot prompt to integrate the syntactic, lexical, elaborative, and content simplification techniques, and generate simplified AR text for head-worn displays. Results from a 16-user empirical study showed that ARTiST lightens the cognitive load and improves performance significantly over both unmodified text and text modified via traditional methods. Our work constitutes a step towards automating the optimization of batch text data for readability and performance in augmented reality.
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-24
Open AccessARTiST: Automated Text Simplification for Task Guidance in Augmented Reality
Text presented in augmented reality provides in-situ, real-time information for users. However, this content can be challenging to apprehend quickly when engaging in cognitively demanding AR tasks, especially when it is presented on a head-mounted display. We propose ARTiST, an automatic text simplification system that uses a few-shot prompt and GPT-3 models to specifically optimize the text length and semantic content for augmented reality. Developed out of a formative study that included seven users and three experts, our system combines a customized error calibration model with a few-shot prompt to integrate the syntactic, lexical, elaborative, and content simplification techniques, and generate simplified AR text for head-worn displays. Results from a 16-user empirical study showed that ARTiST lightens the cognitive load and improves performance significantly over both unmodified text and text modified via traditional methods. Our work constitutes a step towards automating the optimization of batch text data for readability and performance in augmented reality.
- 3 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. PP (10772626) , 1-11
Socrates: Data Story Generation via Adaptive Machine-Guided Elicitation of User Feedback
Visual data stories can effectively convey insights from data, yet their creation often necessitates intricate data exploration, insight discovery, narrative organization, and customization to meet the communication objectives of the storyteller. Existing automated data storytelling techniques, however, tend to overlook the importance of user customization during the data story authoring process, limiting the system's ability to create tailored narratives that reflect the user's intentions. We present a novel data story generation workflow that leverages adaptive machine-guided elicitation of user feedback to customize the story. Our approach employs an adaptive plug-in module for existing story generation systems, which incorporates user feedback through interactive questioning based on the conversation history and dataset. This adaptability refines the system's understanding of the user's intentions, ensuring the final narrative aligns with their goals. We demonstrate the feasibility of our approach through the implementation of an interactive prototype: Socrates. Through a quantitative user study with 18 participants that compares our method to a state-of-the-art data story generation algorithm, we show that Socrates produces more relevant stories with a larger overlap of insights compared to human-generated stories. We also demonstrate the usability of Socrates via interviews with three data analysts and highlight areas of future work.
- 3 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. PP (10772626) , 1-11
Socrates: Data Story Generation via Adaptive Machine-Guided Elicitation of User Feedback
Visual data stories can effectively convey insights from data, yet their creation often necessitates intricate data exploration, insight discovery, narrative organization, and customization to meet the communication objectives of the storyteller. Existing automated data storytelling techniques, however, tend to overlook the importance of user customization during the data story authoring process, limiting the system's ability to create tailored narratives that reflect the user's intentions. We present a novel data story generation workflow that leverages adaptive machine-guided elicitation of user feedback to customize the story. Our approach employs an adaptive plug-in module for existing story generation systems, which incorporates user feedback through interactive questioning based on the conversation history and dataset. This adaptability refines the system's understanding of the user's intentions, ensuring the final narrative aligns with their goals. We demonstrate the feasibility of our approach through the implementation of an interactive prototype: Socrates. Through a quantitative user study with 18 participants that compares our method to a state-of-the-art data story generation algorithm, we show that Socrates produces more relevant stories with a larger overlap of insights compared to human-generated stories. We also demonstrate the usability of Socrates via interviews with three data analysts and highlight areas of future work.
- 2 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 30 (1) , 1313-1323
ARGUS: Visualization of AI-Assisted Task Guidance in AR
The concept of augmented reality (AR) assistants has captured the human imagination for decades, becoming a staple of modern science fiction. To pursue this goal, it is necessary to develop artificial intelligence (AI)-based methods that simultaneously perceive the 3D environment, reason about physical tasks, and model the performer, all in real-time. Within this framework, a wide variety of sensors are needed to generate data across different modalities, such as audio, video, depth, speech, and time-of-flight. The required sensors are typically part of the AR headset, providing performer sensing and interaction through visual, audio, and haptic feedback. AI assistants not only record the performer as they perform activities, but also require machine learning (ML) models to understand and assist the performer as they interact with the physical world. Therefore, developing such assistants is a challenging task. We propose ARGUS, a visual analytics system to support the development of intelligent AR assistants. Our system was designed as part of a multi-year-long collaboration between visualization researchers and ML and AR experts. This co-design process has led to advances in the visualization of ML in AR. Our system allows for online visualization of object, action, and step detection as well as offline analysis of previously recorded AR sessions. It visualizes not only the multimodal sensor data streams but also the output of the ML models. This allows developers to gain insights into the performer activities as well as the ML models, helping them troubleshoot, improve, and fine-tune the components of the AR assistant.
- 2 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 30 (1) , 1313-1323
ARGUS: Visualization of AI-Assisted Task Guidance in AR
The concept of augmented reality (AR) assistants has captured the human imagination for decades, becoming a staple of modern science fiction. To pursue this goal, it is necessary to develop artificial intelligence (AI)-based methods that simultaneously perceive the 3D environment, reason about physical tasks, and model the performer, all in real-time. Within this framework, a wide variety of sensors are needed to generate data across different modalities, such as audio, video, depth, speech, and time-of-flight. The required sensors are typically part of the AR headset, providing performer sensing and interaction through visual, audio, and haptic feedback. AI assistants not only record the performer as they perform activities, but also require machine learning (ML) models to understand and assist the performer as they interact with the physical world. Therefore, developing such assistants is a challenging task. We propose ARGUS, a visual analytics system to support the development of intelligent AR assistants. Our system was designed as part of a multi-year-long collaboration between visualization researchers and ML and AR experts. This co-design process has led to advances in the visualization of ML in AR. Our system allows for online visualization of object, action, and step detection as well as offline analysis of previously recorded AR sessions. It visualizes not only the multimodal sensor data streams but also the output of the ML models. This allows developers to gain insights into the performer activities as well as the ML models, helping them troubleshoot, improve, and fine-tune the components of the AR assistant.