JWJingying Wang

Jingying Wang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Jingying Wang has a total of 33 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    From What Is in Textbooks to What Is in Experts: Human-AI Systems for Externalizing Surgical Expertise

    by Jingying Wang

    Surgical expertise spans more than what textbooks can include. Trainees must acquire explicit knowledge, such as operative decision-making; psychomotor skills accumulated through thousands of hours of practice; and attentional skills, such as knowing what to attend to, which even experts often struggle to articulate. Current training resources address the explicit level well but offer little support for these tacit levels, while access to expert time in the operating room continues to shrink. My dissertation develops human–AI systems that externalize surgical expertise across these three increasingly tacit levels. Surgment and SurgGraph use computer vision to turn surgical recordings into visual exercises with feedback. eXplainMR teaches ultrasonography in mixed reality, explaining why each movement is recommended to help trainees build three-dimensional mental models. SurgGaze enables accurate gaze capture in the operating room through implicit calibration, revealing how experts allocate attention, including strategies such as focus and anticipation. I plan to integrate these levels into residency training through post-operative debriefing that reviews decision-making, action efficiency, and attention allocation in one loop.

  • Open Access

    ReadGaze: Query the Gaze, Shape the Debrief

    by Haoting Gao, Kapotaksha Das, Liang Zhang, Jingying Wang, Mohamed Abouelenien, Michael A Cole, James M. Cooke, Chen Chen, Vitaliy Popov

  • Open Access

    eXplainMR: Generating Real-time Textual and Visual eXplanations to Facilitate UltraSonography Learning in MR

    by Jingying Wang, Jingjing Zhang, Juana Nicoll Capizzano, Matthew Sigakis, Xu Wang, Vitaliy Popov


  • Open Access

    Surgment: Segmentation-enabled Semantic Search and Creation of Visual Question and Feedback to Support Video-Based Surgery Learning

    by Jingying Wang, Haoran Tang, Taylor Kantor, Tandis Soltani, Vitaliy Popov, Xu Wang

    Videos are prominent learning materials to prepare surgical trainees before they enter the operating room (OR). In this work, we explore techniques to enrich the video-based surgery learning experience. We propose Surgment, a system that helps expert surgeons create exercises with feedback based on surgery recordings. Surgment is powered by a few-shot-learning-based pipeline (SegGPT+SAM) to segment surgery scenes, achieving an accuracy of 92%. The segmentation pipeline enables functionalities to create visual questions and feedback desired by surgeons from a formative study. Surgment enables surgeons to 1) retrieve frames of interest through sketches, and 2) design exercises that target specific anatomical components and offer visual feedback. In an evaluation study with 11 surgeons, participants applauded the search-by-sketch approach for identifying frames of interest and found the resulting image-based questions and feedback to be of high educational value.


  • Open Access

    Looking Together ≠ Seeing the Same Thing: Understanding Surgeons' Visual Needs During Intra-operative Coordination and Instruction

    by Vitaliy Popov, Xinyue Chen, Jingying Wang, Michael Kemp, Gurjit Sandhu, Taylor Kantor, Natalie Mateju, Xu Wang

    Shared gaze visualizations have been found to enhance collaboration and communication outcomes in diverse HCI scenarios including computer supported collaborative work and learning contexts. Given the importance of gaze in surgery operations, especially when a surgeon trainer and trainee need to coordinate their actions, research on the use of gaze to facilitate intra-operative coordination and instruction has been limited and shows mixed implications. We performed a field observation of 8 surgeries and an interview study with 14 surgeons to understand their visual needs during operations, informing ways to leverage and augment gaze to enhance intra-operative coordination and instruction. We found that trainees have varying needs in receiving visual guidance which are often unfulfilled by the trainers’ instructions. It is critical for surgeons to control the timing of the gaze-based visualizations and effectively interpret gaze data. We suggest overlay technologies, e.g., gaze-based summaries and depth sensing, to augment raw gaze in support of surgical coordination and instruction.


  • Open Access

    SketchSearch: Fine-tuning Reference Maps to Create Exercises In Support of Video-based Learning for Surgeons

    by Jingying Wang, Vitaliy Popov, Xu Wang

    Video-based surgical coaching involves mentors reviewing surgery footage with trainees. Although effective, its use is sporadic due to time constraints. We propose AI-augmented coaching through SketchSearch, allowing experts to create exercises with automated feedback in surgery videos for self-learning. Surgeons often seek specific scenes for teaching, relying on visual cues. SketchSearch simplifies this through a three-step process: key frame extraction, template reference maps creation via image segmentation, and fine-tuning for frame retrieval.


  • Open Access

    Real-time Facial Animation for 3D Stylized Character with Emotion Dynamics

    by Ye Pan, Ruisi Zhang, Jingying Wang, Yu Ding, Kenny Mitchell

    Our aim is to improve animation production techniques' efficiency and effectiveness. We present two real-time solutions which drive character expressions in a geometrically consistent and perceptually valid way. Our first solution combines keyframe animation techniques with machine learning models. We propose a 3D emotion transfer network makes use of a 2D human image to generate a stylized 3D rig parameter. Our second solution combines blendshape-based motion capture animation techniques with machine learning models. We propose a blendshape adaption network which generates the character rig parameter motions with geometric consistency and temporally stability. We demonstrate the effectiveness of our system by comparing it to a commercial product Faceware. Results reveal that ratings of the recognition, intensity, and attractiveness of expressions depicted for animated characters via our systems are statistically higher than Faceware. Our results may be implemented into the animation pipeline, supporting animators to create expressions more rapidly and precisely.


  • ConeSpeech: Exploring Directional Speech Interaction for Multi-Person Remote Communication in Virtual Reality

    by Yukang Yan, Haohua Liu, Yingtian Shi, Jingying Wang, Ruici Guo, Zisu Li, Xuhai Xu, Chun Yu, Yuntao Wang, Yuanchun Shi

    Remote communication is essential for efficient collaboration among people at different locations. We present ConeSpeech, a virtual reality (VR) based multi-user remote communication technique, which enables users to selectively speak to target listeners without distracting bystanders. With ConeSpeech, the user looks at the target listener and only in a cone-shaped area in the direction can the listeners hear the speech. This manner alleviates the disturbance to and avoids overhearing from surrounding irrelevant people. Three featured functions are supported, directional speech delivery, size-adjustable delivery range, and multiple delivery areas, to facilitate speaking to more than one listener and to listeners spatially mixed up with bystanders. We conducted a user study to determine the modality to control the cone-shaped delivery area. Then we implemented the technique and evaluated its performance in three typical multi-user communication tasks by comparing it to two baseline methods. Results show that ConeSpeech balanced the convenience and flexibility of voice communication.