Jing Qian
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Jing Qian has a total of 46 co-authors. Here are the top co-authors by number of shared publications:
- Guande Wu
Tandon School of Engineering, New York University, New York City, New York, USA
7 shared publications - Joao Rulff
New York University, New York, NY, USA
5 shared publications - Sonia Castelo
New York University, New York, New York, USA
5 shared publications - Claudio Silva
New York University
4 shared publications - Shaoyu Chen
New York University, New York, NY, USA
4 shared publications - Bea Steers
New York University, New York, NY, USA
3 shared publications
Latest Publications
- Open Access
- 4 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 141-150
CHORUS: Designing Human–AI Multi-Agent Collaboration for Professional Translators
- 4 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 141-150
Open AccessCHORUS: Designing Human–AI Multi-Agent Collaboration for Professional Translators
- Open Access
- 7 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-9
InsightAR: A Tool for Multi-modal Summarization and Interactive Analysis of AR-based Egocentric Task Videos
The growing use of head-mounted AR devices has produced a surge of egocentric recordings of procedural tasks—step-by-step activities such as medical procedures—creating rich opportunities for performance analysis. Yet current analyses are time-consuming, requiring substantial manual inspection to identify events, actions, and objects from videos, with little automated support for surfacing insights or errors. We introduce InsightAR, a visual analytics tool that integrates event detection, action recognition, and multi-modal language models to create video summaries capturing key errors and insights. Its interface further enables users to refine generated insights and errors collaboratively. A user study (N=16) found that InsightAR identifies significantly more errors and marginally significant insight numbers than the baseline. Our work demonstrated the feasibility and potential benefits of future human-AI collaborative video analysis.
- 7 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-9
Open AccessInsightAR: A Tool for Multi-modal Summarization and Interactive Analysis of AR-based Egocentric Task Videos
The growing use of head-mounted AR devices has produced a surge of egocentric recordings of procedural tasks—step-by-step activities such as medical procedures—creating rich opportunities for performance analysis. Yet current analyses are time-consuming, requiring substantial manual inspection to identify events, actions, and objects from videos, with little automated support for surfacing insights or errors. We introduce InsightAR, a visual analytics tool that integrates event detection, action recognition, and multi-modal language models to create video summaries capturing key errors and insights. Its interface further enables users to refine generated insights and errors collaboratively. A user study (N=16) found that InsightAR identifies significantly more errors and marginally significant insight numbers than the baseline. Our work demonstrated the feasibility and potential benefits of future human-AI collaborative video analysis.
- Open Access
- 9 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-12
Cam-2-Cam: Exploring the Design Space of Dual-Camera Interactions for Smartphone-based Augmented Reality
- 9 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-12
Open AccessCam-2-Cam: Exploring the Design Space of Dual-Camera Interactions for Smartphone-based Augmented Reality
- Top Cited PapersOpen Access
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-24
Satori 悟り: Towards Proactive AR Assistant with Belief-Desire-Intention User Modeling
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-24
Top Cited PapersOpen AccessSatori 悟り: Towards Proactive AR Assistant with Belief-Desire-Intention User Modeling
- 10 March 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Graphics and Applications
- Vol. 45 (1) , 28-42
Design and Implementation of the Transparent, Interpretable, and Multimodal (TIM) AR Personal Assistant
The concept of an AI assistant for task guidance is rapidly shifting from a science fiction staple to an impending reality. Such a system is inherently complex, requiring models for perceptual grounding, attention, and reasoning, an intuitive interface that adapts to the performer's needs, and the orchestration of data streams from many sensors. Moreover, all data acquired by the system must be readily available for post-hoc analysis to enable developers to understand performer behavior and quickly detect failures. We introduce TIM, the first end-to-end AI-enabled task guidance system in augmented reality which is capable of detecting both the user and scene as well as providing adaptable, just-in-time feedback. We discuss the system challenges and propose design solutions. We also demonstrate how TIM adapts to domain applications with varying needs, highlighting how the system components can be customized for each scenario.
- 10 March 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Graphics and Applications
- Vol. 45 (1) , 28-42
Design and Implementation of the Transparent, Interpretable, and Multimodal (TIM) AR Personal Assistant
The concept of an AI assistant for task guidance is rapidly shifting from a science fiction staple to an impending reality. Such a system is inherently complex, requiring models for perceptual grounding, attention, and reasoning, an intuitive interface that adapts to the performer's needs, and the orchestration of data streams from many sensors. Moreover, all data acquired by the system must be readily available for post-hoc analysis to enable developers to understand performer behavior and quickly detect failures. We introduce TIM, the first end-to-end AI-enabled task guidance system in augmented reality which is capable of detecting both the user and scene as well as providing adaptable, just-in-time feedback. We discuss the system challenges and propose design solutions. We also demonstrate how TIM adapts to domain applications with varying needs, highlighting how the system components can be customized for each scenario.
- Open Access
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-24
ARTiST: Automated Text Simplification for Task Guidance in Augmented Reality
Text presented in augmented reality provides in-situ, real-time information for users. However, this content can be challenging to apprehend quickly when engaging in cognitively demanding AR tasks, especially when it is presented on a head-mounted display. We propose ARTiST, an automatic text simplification system that uses a few-shot prompt and GPT-3 models to specifically optimize the text length and semantic content for augmented reality. Developed out of a formative study that included seven users and three experts, our system combines a customized error calibration model with a few-shot prompt to integrate the syntactic, lexical, elaborative, and content simplification techniques, and generate simplified AR text for head-worn displays. Results from a 16-user empirical study showed that ARTiST lightens the cognitive load and improves performance significantly over both unmodified text and text modified via traditional methods. Our work constitutes a step towards automating the optimization of batch text data for readability and performance in augmented reality.
- 11 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-24
Open AccessARTiST: Automated Text Simplification for Task Guidance in Augmented Reality
Text presented in augmented reality provides in-situ, real-time information for users. However, this content can be challenging to apprehend quickly when engaging in cognitively demanding AR tasks, especially when it is presented on a head-mounted display. We propose ARTiST, an automatic text simplification system that uses a few-shot prompt and GPT-3 models to specifically optimize the text length and semantic content for augmented reality. Developed out of a formative study that included seven users and three experts, our system combines a customized error calibration model with a few-shot prompt to integrate the syntactic, lexical, elaborative, and content simplification techniques, and generate simplified AR text for head-worn displays. Results from a 16-user empirical study showed that ARTiST lightens the cognitive load and improves performance significantly over both unmodified text and text modified via traditional methods. Our work constitutes a step towards automating the optimization of batch text data for readability and performance in augmented reality.
- 2 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 30 (1) , 1313-1323
ARGUS: Visualization of AI-Assisted Task Guidance in AR
The concept of augmented reality (AR) assistants has captured the human imagination for decades, becoming a staple of modern science fiction. To pursue this goal, it is necessary to develop artificial intelligence (AI)-based methods that simultaneously perceive the 3D environment, reason about physical tasks, and model the performer, all in real-time. Within this framework, a wide variety of sensors are needed to generate data across different modalities, such as audio, video, depth, speech, and time-of-flight. The required sensors are typically part of the AR headset, providing performer sensing and interaction through visual, audio, and haptic feedback. AI assistants not only record the performer as they perform activities, but also require machine learning (ML) models to understand and assist the performer as they interact with the physical world. Therefore, developing such assistants is a challenging task. We propose ARGUS, a visual analytics system to support the development of intelligent AR assistants. Our system was designed as part of a multi-year-long collaboration between visualization researchers and ML and AR experts. This co-design process has led to advances in the visualization of ML in AR. Our system allows for online visualization of object, action, and step detection as well as offline analysis of previously recorded AR sessions. It visualizes not only the multimodal sensor data streams but also the output of the ML models. This allows developers to gain insights into the performer activities as well as the ML models, helping them troubleshoot, improve, and fine-tune the components of the AR assistant.
- 2 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 30 (1) , 1313-1323
ARGUS: Visualization of AI-Assisted Task Guidance in AR
The concept of augmented reality (AR) assistants has captured the human imagination for decades, becoming a staple of modern science fiction. To pursue this goal, it is necessary to develop artificial intelligence (AI)-based methods that simultaneously perceive the 3D environment, reason about physical tasks, and model the performer, all in real-time. Within this framework, a wide variety of sensors are needed to generate data across different modalities, such as audio, video, depth, speech, and time-of-flight. The required sensors are typically part of the AR headset, providing performer sensing and interaction through visual, audio, and haptic feedback. AI assistants not only record the performer as they perform activities, but also require machine learning (ML) models to understand and assist the performer as they interact with the physical world. Therefore, developing such assistants is a challenging task. We propose ARGUS, a visual analytics system to support the development of intelligent AR assistants. Our system was designed as part of a multi-year-long collaboration between visualization researchers and ML and AR experts. This co-design process has led to advances in the visualization of ML in AR. Our system allows for online visualization of object, action, and step detection as well as offline analysis of previously recorded AR sessions. It visualizes not only the multimodal sensor data streams but also the output of the ML models. This allows developers to gain insights into the performer activities as well as the ML models, helping them troubleshoot, improve, and fine-tune the components of the AR assistant.
- Open Access
ARGUS: Visualization of AI-Assisted Task Guidance in AR
The concept of augmented reality (AR) assistants has captured the human imagination for decades, becoming a staple of modern science fiction. To pursue this goal, it is necessary to develop artificial intelligence (AI)-based methods that simultaneously perceive the 3D environment, reason about physical tasks, and model the performer, all in real-time. Within this framework, a wide variety of sensors are needed to generate data across different modalities, such as audio, video, depth, speech, and time-of-flight. The required sensors are typically part of the AR headset, providing performer sensing and interaction through visual, audio, and haptic feedback. AI assistants not only record the performer as they perform activities, but also require machine learning (ML) models to understand and assist the performer as they interact with the physical world. Therefore, developing such assistants is a challenging task. We propose ARGUS, a visual analytics system to support the development of intelligent AR assistants. Our system was designed as part of a multi year-long collaboration between visualization researchers and ML and AR experts. This co-design process has led to advances in the visualization of ML in AR. Our system allows for online visualization of object, action, and step detection as well as offline analysis of previously recorded AR sessions. It visualizes not only the multimodal sensor data streams but also the output of the ML models. This allows developers to gain insights into the performer activities as well as the ML models, helping them troubleshoot, improve, and fine tune the components of the AR assistant.
Open AccessARGUS: Visualization of AI-Assisted Task Guidance in AR
The concept of augmented reality (AR) assistants has captured the human imagination for decades, becoming a staple of modern science fiction. To pursue this goal, it is necessary to develop artificial intelligence (AI)-based methods that simultaneously perceive the 3D environment, reason about physical tasks, and model the performer, all in real-time. Within this framework, a wide variety of sensors are needed to generate data across different modalities, such as audio, video, depth, speech, and time-of-flight. The required sensors are typically part of the AR headset, providing performer sensing and interaction through visual, audio, and haptic feedback. AI assistants not only record the performer as they perform activities, but also require machine learning (ML) models to understand and assist the performer as they interact with the physical world. Therefore, developing such assistants is a challenging task. We propose ARGUS, a visual analytics system to support the development of intelligent AR assistants. Our system was designed as part of a multi year-long collaboration between visualization researchers and ML and AR experts. This co-design process has led to advances in the visualization of ML in AR. Our system allows for online visualization of object, action, and step detection as well as offline analysis of previously recorded AR sessions. It visualizes not only the multimodal sensor data streams but also the output of the ML models. This allows developers to gain insights into the performer activities as well as the ML models, helping them troubleshoot, improve, and fine tune the components of the AR assistant.
- Open Access
- 27 March 2023
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 7 (1) , 1-26
FocalPoint
We propose FocalPoint, a direct manipulation technique in smartphone augmented reality (AR) for selecting small densely-packed objects within reach, a fundamental yet challenging task in AR due to the required accuracy and precision. FocalPoint adaptively and continuously updates a cylindrical geometry for selection disambiguation based on the user's selection history and hand movements. This design is informed by a preliminary study which revealed that participants preferred selecting objects appearing in particular regions of the screen. We evaluate FocalPoint against a baseline direct manipulation technique in a 12-participant study with two tasks: selecting a 3 mm wide target from a pile of cubes and virtually decorating a house with LEGO pieces. FocalPoint was three times as accurate for selecting the correct object and 5.5 seconds faster on average; participants using FocalPoint decorated their houses more and were more satisfied with the result. We further demonstrate the finer control enabled by FocalPoint in example applications of robot repair, 3D modeling, and neural network visualizations.
- 27 March 2023
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 7 (1) , 1-26
Open AccessFocalPoint
We propose FocalPoint, a direct manipulation technique in smartphone augmented reality (AR) for selecting small densely-packed objects within reach, a fundamental yet challenging task in AR due to the required accuracy and precision. FocalPoint adaptively and continuously updates a cylindrical geometry for selection disambiguation based on the user's selection history and hand movements. This design is informed by a preliminary study which revealed that participants preferred selecting objects appearing in particular regions of the screen. We evaluate FocalPoint against a baseline direct manipulation technique in a 12-participant study with two tasks: selecting a 3 mm wide target from a pile of cubes and virtually decorating a house with LEGO pieces. FocalPoint was three times as accurate for selecting the correct object and 5.5 seconds faster on average; participants using FocalPoint decorated their houses more and were more satisfied with the result. We further demonstrate the finer control enabled by FocalPoint in example applications of robot repair, 3D modeling, and neural network visualizations.
- Open Access
- 19 March 2023
- Published by Association for Computing Machinery (ACM)
- p. 449-453
irchiver: A Full-Resolution Personal Web Archive for Users and Researchers
Irchiver is a personal web archive tool for users, which also provided an opportunity for information retrieval researchers to access naturalistic long-term browsing histories. It offers users the ability to capture and search their web archives, which are stored in full-resolution images on their local filesystem. Researchers can request these files, or develop a plugin that triggers when a new image is captured. irchiver runs as a Windows background process using a hook and capture technique that works on all 5 major desktop browsers. The first author has been using irchiver for 12 months, encountering unexpected benefits in archive retrieval, change detection, search functionality, and content recovery.
- 19 March 2023
- Published by Association for Computing Machinery (ACM)
- p. 449-453
Open Accessirchiver: A Full-Resolution Personal Web Archive for Users and Researchers
Irchiver is a personal web archive tool for users, which also provided an opportunity for information retrieval researchers to access naturalistic long-term browsing histories. It offers users the ability to capture and search their web archives, which are stored in full-resolution images on their local filesystem. Researchers can request these files, or develop a plugin that triggers when a new image is captured. irchiver runs as a Windows background process using a hook and capture technique that works on all 5 major desktop browsers. The first author has been using irchiver for 12 months, encountering unexpected benefits in archive retrieval, change detection, search functionality, and content recovery.