Yonghao Long
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Yonghao Long has a total of 102 co-authors. Here are the top co-authors by number of shared publications:
- Qi Dou
The Chinese University of Hong Kong
13 shared publications - Peter Kazanzides
Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD 21218, USA
3 shared publications Qi Dou
Chinese Univ Hong Kong, Dept Comp Sci & Engn, Hong Kong, Peoples R China
3 shared publications- Yun-Hui Liu
The Chinese University of Hong Kong
3 shared publications - Chengkun Li
Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China
2 shared publications - Dong Sun
Department of Biomedical Engineering, City University of Hong Kong, Kowloon, Hong Kong, China
2 shared publications
Latest Publications
- 1 December 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11  (12) , 15084-15091
Execution-Level Correction for Surgical Robot Automation via Residual Policy Learning From Human Interventions
- 1 December 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11  (12) , 15084-15091
Execution-Level Correction for Surgical Robot Automation via Residual Policy Learning From Human Interventions
- Open Access
- 1 July 2026
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 21  (7) , 1639-1647
ARport: an augmented reality system for markerless image-guided port placement in robotic surgery
Purpose: Precise port placement is a critical step in robot-assisted surgery, where port configuration influences both visual access to the operative field and instrument maneuverability. To bridge the gap between preoperative planning and intraoperative execution, we present ARport, an augmented reality (AR) system that automatically maps pre-planned trocar layouts onto the patient’s body surface , providing intuitive spatial guidance during surgical preparation. Methods: ARport, implemented on an optical see-through head-mounted display (OST-HMD), operates without any external sensors or markers, simplifying setup and enhancing workflow integration. It reconstructs the operative scene from RGB, depth, and pose data captured by the OST-HMD, extracts the patient’s body surface using a foundation model, and performs surface-based markerless registration to align preoperative anatomical models to the extracted patient’s body surface, enabling in situ visualization of planned trocar layouts. A demonstration video illustrating the overall workflow is available online. Results: In full-scale human phantom experiments, ARport accurately overlaid pre-planned trocar sites onto the physical phantom, achieving consistent spatial correspondence between virtual plans and real anatomy. Conclusion: ARport provides a fully marker-free and hardware-minimal solution for visualizing preoperative trocar plans directly on the patient’s body surface. The system facilitates efficient intraoperative setup and demonstrates potential for seamless integration into routine clinical workflows.
- 1 July 2026
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 21  (7) , 1639-1647
Open AccessARport: an augmented reality system for markerless image-guided port placement in robotic surgery
Purpose: Precise port placement is a critical step in robot-assisted surgery, where port configuration influences both visual access to the operative field and instrument maneuverability. To bridge the gap between preoperative planning and intraoperative execution, we present ARport, an augmented reality (AR) system that automatically maps pre-planned trocar layouts onto the patient’s body surface , providing intuitive spatial guidance during surgical preparation. Methods: ARport, implemented on an optical see-through head-mounted display (OST-HMD), operates without any external sensors or markers, simplifying setup and enhancing workflow integration. It reconstructs the operative scene from RGB, depth, and pose data captured by the OST-HMD, extracts the patient’s body surface using a foundation model, and performs surface-based markerless registration to align preoperative anatomical models to the extracted patient’s body surface, enabling in situ visualization of planned trocar layouts. A demonstration video illustrating the overall workflow is available online. Results: In full-scale human phantom experiments, ARport accurately overlaid pre-planned trocar sites onto the physical phantom, achieving consistent spatial correspondence between virtual plans and real anatomy. Conclusion: ARport provides a fully marker-free and hardware-minimal solution for visualizing preoperative trocar plans directly on the patient’s body surface. The system facilitates efficient intraoperative setup and demonstrates potential for seamless integration into routine clinical workflows.
- 1 June 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11  (6) , 6568-6575
Self-Supervised Adaptive Transformer for Surgical Step Recognition in Robotic-Assisted Radical Prostatectomy
The automatic recognition of surgical steps is essential for enhancing situational awareness and workflow automation in robotic-assisted surgery. However, existing vision-based approaches exhibit limitations in effectively leveraging rich spatial-temporal information from surgical videos, particularly when addressing generalization challenges across different clinical centers, surgeons, and patient populations. Current methods struggle with domain adaptation when deployed in diverse real-world settings due to substantial variations in surgical techniques, anatomical presentations, imaging conditions (such as lighting and white balance), and data preprocessing protocols. To address these limitations, we propose ProstaFormer (Prostatectomy Steps Transformer with Adaptive Feature Fusion), a framework that integrates a vision learner pre-trained via MAE with transformer-based temporal features through an adaptive fusion mechanism. Our approach intelligently combines spatial and temporal representations using position-aware attention weighting, enabling robust recognition of complex surgical workflow patterns across diverse clinical environments. Furthermore, we incorporate a diffusion-based Temporal Adaptation module to rectify domain-specific temporal order differences. We curate and annotate the comprehensive RPSteps dataset and conduct extensive experiments on both GraSP and RPSteps datasets. ProstaFormer consistently outperforms strong baselines, demonstrating improved generalization to different hospitals, surgeons, and patient populations, as well as superior robustness to image degradation. These results highlight the potential of adaptive feature fusion, temporal adaptation, and self-supervised visual pre-training for advancing intelligent robotic-assisted surgical workflow recognition. The source code and supplementary materials (including detailed pre-training setups and extended experimental results) are available at https://github.com/ProstaFormer/ProstaFormer.git.
- 1 June 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11  (6) , 6568-6575
Self-Supervised Adaptive Transformer for Surgical Step Recognition in Robotic-Assisted Radical Prostatectomy
The automatic recognition of surgical steps is essential for enhancing situational awareness and workflow automation in robotic-assisted surgery. However, existing vision-based approaches exhibit limitations in effectively leveraging rich spatial-temporal information from surgical videos, particularly when addressing generalization challenges across different clinical centers, surgeons, and patient populations. Current methods struggle with domain adaptation when deployed in diverse real-world settings due to substantial variations in surgical techniques, anatomical presentations, imaging conditions (such as lighting and white balance), and data preprocessing protocols. To address these limitations, we propose ProstaFormer (Prostatectomy Steps Transformer with Adaptive Feature Fusion), a framework that integrates a vision learner pre-trained via MAE with transformer-based temporal features through an adaptive fusion mechanism. Our approach intelligently combines spatial and temporal representations using position-aware attention weighting, enabling robust recognition of complex surgical workflow patterns across diverse clinical environments. Furthermore, we incorporate a diffusion-based Temporal Adaptation module to rectify domain-specific temporal order differences. We curate and annotate the comprehensive RPSteps dataset and conduct extensive experiments on both GraSP and RPSteps datasets. ProstaFormer consistently outperforms strong baselines, demonstrating improved generalization to different hospitals, surgeons, and patient populations, as well as superior robustness to image degradation. These results highlight the potential of adaptive feature fusion, temporal adaptation, and self-supervised visual pre-training for advancing intelligent robotic-assisted surgical workflow recognition. The source code and supplementary materials (including detailed pre-training setups and extended experimental results) are available at https://github.com/ProstaFormer/ProstaFormer.git.
- Open Access
ARport: An Augmented Reality System for Markerless Image-Guided Port Placement in Robotic Surgery
Purpose: Precise port placement is a critical step in robot-assisted surgery, where port configuration influences both visual access to the operative field and instrument maneuverability. To bridge the gap between preoperative planning and intraoperative execution, we present ARport, an augmented reality (AR) system that automatically maps pre-planned trocar layouts onto the patient's body surface, providing intuitive spatial guidance during surgical preparation. Methods: ARport, implemented on an optical see-through head-mounted display (OST-HMD), operates without any external sensors or markers, simplifying setup and enhancing workflow integration. It reconstructs the operative scene from RGB, depth, and pose data captured by the OST-HMD, extracts the patient's body surface using a foundation model, and performs surface-based markerless registration to align preoperative anatomical models to the extracted patient's body surface, enabling in-situ visualization of planned trocar layouts. A demonstration video illustrating the overall workflow is available online. Results: In full-scale human-phantom experiments, ARport accurately overlaid pre-planned trocar sites onto the physical phantom, achieving consistent spatial correspondence between virtual plans and real anatomy. Conclusion: ARport provides a fully marker-free and hardware-minimal solution for visualizing preoperative trocar plans directly on the patient's body surface. The system facilitates efficient intraoperative setup and demonstrates potential for seamless integration into routine clinical workflows.
Open AccessARport: An Augmented Reality System for Markerless Image-Guided Port Placement in Robotic Surgery
Purpose: Precise port placement is a critical step in robot-assisted surgery, where port configuration influences both visual access to the operative field and instrument maneuverability. To bridge the gap between preoperative planning and intraoperative execution, we present ARport, an augmented reality (AR) system that automatically maps pre-planned trocar layouts onto the patient's body surface, providing intuitive spatial guidance during surgical preparation. Methods: ARport, implemented on an optical see-through head-mounted display (OST-HMD), operates without any external sensors or markers, simplifying setup and enhancing workflow integration. It reconstructs the operative scene from RGB, depth, and pose data captured by the OST-HMD, extracts the patient's body surface using a foundation model, and performs surface-based markerless registration to align preoperative anatomical models to the extracted patient's body surface, enabling in-situ visualization of planned trocar layouts. A demonstration video illustrating the overall workflow is available online. Results: In full-scale human-phantom experiments, ARport accurately overlaid pre-planned trocar sites onto the physical phantom, achieving consistent spatial correspondence between virtual plans and real anatomy. Conclusion: ARport provides a fully marker-free and hardware-minimal solution for visualizing preoperative trocar plans directly on the patient's body surface. The system facilitates efficient intraoperative setup and demonstrates potential for seamless integration into routine clinical workflows.
- Top Cited Papers
- 16 July 2025
- Published by American Association for the Advancement of Science (AAAS) in Science Robotics
- Vol. 10  (104) , eadt3093
Surgical embodied intelligence for generalized task autonomy in laparoscopic robot-assisted surgery
Surgical robots capable of autonomously performing various tasks could enhance efficiency and augment human productivity in addressing clinical needs. Although current solutions have automated specific actions within defined contexts, they are challenging to generalize across diverse environments in general surgery. Embodied intelligence enables general-purpose robot learning with applications for daily tasks, yet its application in the medical domain remains limited. We introduced an open-source surgical embodied intelligence simulator for an interactive environment to develop reinforcement learning methods for minimally invasive surgical robots. Using such embodied artificial intelligence, this study further addresses surgical task automation, enabling zero-shot transfer of simulation-trained policies to real-world scenarios. The proposed method encompasses visual parsing, a perceptual regressor, policy learning, and a visual servoing controller, forming a paradigm that combines the advantages of data-driven policy and classic controller. The visual parsing uses stereo depth estimation and image segmentation with a visual foundation model to handle complex scenes. Experiments demonstrated autonomy in seven game-based skill training tasks on the da Vinci Research Kit, with a proof-of-concept study on haptic-assisted skill training as a practical application. Moreover, we conducted automation of five surgical assistive tasks with the Sentire surgical system on ex vivo animal tissues with various scenes, object sizes, instrument types, and illuminations. The learned policies were also validated in a live-animal trial for three tasks in dynamic in vivo surgical environments. We hope this open-source infrastructure, coupled with a general-purpose learning paradigm, will inspire and facilitate future research on embodied intelligence toward autonomous surgical robots.
- 16 July 2025
- Published by American Association for the Advancement of Science (AAAS) in Science Robotics
- Vol. 10  (104) , eadt3093
Top Cited PapersSurgical embodied intelligence for generalized task autonomy in laparoscopic robot-assisted surgery
Surgical robots capable of autonomously performing various tasks could enhance efficiency and augment human productivity in addressing clinical needs. Although current solutions have automated specific actions within defined contexts, they are challenging to generalize across diverse environments in general surgery. Embodied intelligence enables general-purpose robot learning with applications for daily tasks, yet its application in the medical domain remains limited. We introduced an open-source surgical embodied intelligence simulator for an interactive environment to develop reinforcement learning methods for minimally invasive surgical robots. Using such embodied artificial intelligence, this study further addresses surgical task automation, enabling zero-shot transfer of simulation-trained policies to real-world scenarios. The proposed method encompasses visual parsing, a perceptual regressor, policy learning, and a visual servoing controller, forming a paradigm that combines the advantages of data-driven policy and classic controller. The visual parsing uses stereo depth estimation and image segmentation with a visual foundation model to handle complex scenes. Experiments demonstrated autonomy in seven game-based skill training tasks on the da Vinci Research Kit, with a proof-of-concept study on haptic-assisted skill training as a practical application. Moreover, we conducted automation of five surgical assistive tasks with the Sentire surgical system on ex vivo animal tissues with various scenes, object sizes, instrument types, and illuminations. The learned policies were also validated in a live-animal trial for three tasks in dynamic in vivo surgical environments. We hope this open-source infrastructure, coupled with a general-purpose learning paradigm, will inspire and facilitate future research on embodied intelligence toward autonomous surgical robots.
- 10 May 2025
- Published by Elsevier in Medical Image Analysis
- Vol. 103, 103599
Learning dissection trajectories from expert surgical videos via imitation learning with equivariant diffusion
- 10 May 2025
- Published by Elsevier in Medical Image Analysis
- Vol. 103, 103599
Learning dissection trajectories from expert surgical videos via imitation learning with equivariant diffusion
- 6 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9  (10) , 9135-9142
Extended Reality With HMD-Assisted Guidance and Console 3D Overlay for Robotic Surgery Remote Mentoring
The concept of remote-guided surgery has garnered significant attention among researchers as a possible solution to transcend geographic barriers and facilitate the integration of medical resources across diverse regions. However, establishing effective communication channels between remote specialists and surgeons poses an ongoing challenge in this field. In this paper, we propose an extended reality (XR) remote mentoring system for robotic surgery. The system employs an optical see-through head-mounted display (OST-HMD) to provide real-time 3D endoscopic visualization and enable remote mentors to offer surgical guidance through natural hand gestures and virtual surgical instruments. Additionally, a console 3D overlay method is introduced to enhance the depth perception of the console surgeon by stereo occlusion-aware overlaying augmented surgical tools onto the real endoscopic video. The system incorporates the assistant's role in the surgical team by transmitting the assistant's view to the console surgeon and remote mentor, enriching the information in the operating room. Latency and accuracy experiments have proved the efficiency of the system. The user study in a simulated surgical task shows promising performance for optimizing user experience and reducing communication costs.
- 6 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9  (10) , 9135-9142
Extended Reality With HMD-Assisted Guidance and Console 3D Overlay for Robotic Surgery Remote Mentoring
The concept of remote-guided surgery has garnered significant attention among researchers as a possible solution to transcend geographic barriers and facilitate the integration of medical resources across diverse regions. However, establishing effective communication channels between remote specialists and surgeons poses an ongoing challenge in this field. In this paper, we propose an extended reality (XR) remote mentoring system for robotic surgery. The system employs an optical see-through head-mounted display (OST-HMD) to provide real-time 3D endoscopic visualization and enable remote mentors to offer surgical guidance through natural hand gestures and virtual surgical instruments. Additionally, a console 3D overlay method is introduced to enhance the depth perception of the console surgeon by stereo occlusion-aware overlaying augmented surgical tools onto the real endoscopic video. The system incorporates the assistant's role in the surgical team by transmitting the assistant's view to the console surgeon and remote mentor, enriching the information in the operating room. Latency and accuracy experiments have proved the efficiency of the system. The user study in a simulated surgical task shows promising performance for optimizing user experience and reducing communication costs.
- Open Access
- 7 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 43  (12) , 4356-4367
Self-Supervised Cyclic Diffeomorphic Mapping for Soft Tissue Deformation Recovery in Robotic Surgery Scenes
The ability to recover tissue deformation from visual features is fundamental for many robotic surgery applications. This has been a long-standing research topic in computer vision, however, is still unsolved due to complex dynamics of soft tissues when being manipulated by surgical instruments. The ambiguous pixel correspondence caused by homogeneous texture makes achieving dense and accurate tissue tracking even more challenging. In this paper, we propose a novel self-supervised framework to recover tissue deformations from stereo surgical videos. Our approach integrates semantics, cross-frame motion flow, and long-range temporal dependencies to enable the recovered deformations to represent actual tissue dynamics. Moreover, we incorporate diffeomorphic mapping to regularize the warping field to be physically realistic. To comprehensively evaluate our method, we collected stereo surgical video clips containing three types of tissue manipulation (i.e., pushing, dissection and retraction) from two different types of surgeries (i.e., hemicolectomy and mesorectal excision). Our method has achieved impressive results in capturing deformation in 3D mesh, and generalized well across manipulations and surgeries. It also outperforms current state-of-the-art methods on non-rigid registration and optical flow estimation. To the best of our knowledge, this is the first work on self-supervised learning for dense tissue deformation modeling from stereo surgical videos. Our code will be released.
- 7 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 43  (12) , 4356-4367
Open AccessSelf-Supervised Cyclic Diffeomorphic Mapping for Soft Tissue Deformation Recovery in Robotic Surgery Scenes
The ability to recover tissue deformation from visual features is fundamental for many robotic surgery applications. This has been a long-standing research topic in computer vision, however, is still unsolved due to complex dynamics of soft tissues when being manipulated by surgical instruments. The ambiguous pixel correspondence caused by homogeneous texture makes achieving dense and accurate tissue tracking even more challenging. In this paper, we propose a novel self-supervised framework to recover tissue deformations from stereo surgical videos. Our approach integrates semantics, cross-frame motion flow, and long-range temporal dependencies to enable the recovered deformations to represent actual tissue dynamics. Moreover, we incorporate diffeomorphic mapping to regularize the warping field to be physically realistic. To comprehensively evaluate our method, we collected stereo surgical video clips containing three types of tissue manipulation (i.e., pushing, dissection and retraction) from two different types of surgeries (i.e., hemicolectomy and mesorectal excision). Our method has achieved impressive results in capturing deformation in 3D mesh, and generalized well across manipulations and surgeries. It also outperforms current state-of-the-art methods on non-rigid registration and optical flow estimation. To the best of our knowledge, this is the first work on self-supervised learning for dense tissue deformation modeling from stereo surgical videos. Our code will be released.
- Top Cited PapersOpen Access
- 21 October 2023
- Published by Springer Nature in Nature Communications
- Vol. 14  (1) , 1-14
Intelligent surgical workflow recognition for endoscopic submucosal dissection with real-time animal study
Recent advancements in artificial intelligence have witnessed human-level performance; however, AI-enabled cognitive assistance for therapeutic procedures has not been fully explored nor pre-clinically validated. Here we propose AI-Endo, an intelligent surgical workflow recognition suit, for endoscopic submucosal dissection (ESD). Our AI-Endo is trained on high-quality ESD cases from an expert endoscopist, covering a decade time expansion and consisting of 201,026 labeled frames. The learned model demonstrates outstanding performance on validation data, including cases from relatively junior endoscopists with various skill levels, procedures conducted with different endoscopy systems and therapeutic skills, and cohorts from international multi-centers. Furthermore, we integrate our AI-Endo with the Olympus endoscopic system and validate the AI-enabled cognitive assistance system with animal studies in live ESD training sessions. Dedicated data analysis from surgical phase recognition results is summarized in an automatically generated report for skill assessment.
- 21 October 2023
- Published by Springer Nature in Nature Communications
- Vol. 14  (1) , 1-14
Top Cited PapersOpen AccessIntelligent surgical workflow recognition for endoscopic submucosal dissection with real-time animal study
Recent advancements in artificial intelligence have witnessed human-level performance; however, AI-enabled cognitive assistance for therapeutic procedures has not been fully explored nor pre-clinically validated. Here we propose AI-Endo, an intelligent surgical workflow recognition suit, for endoscopic submucosal dissection (ESD). Our AI-Endo is trained on high-quality ESD cases from an expert endoscopist, covering a decade time expansion and consisting of 201,026 labeled frames. The learned model demonstrates outstanding performance on validation data, including cases from relatively junior endoscopists with various skill levels, procedures conducted with different endoscopy systems and therapeutic skills, and cohorts from international multi-centers. Furthermore, we integrate our AI-Endo with the Olympus endoscopic system and validate the AI-enabled cognitive assistance system with animal studies in live ESD training sessions. Dedicated data analysis from surgical phase recognition results is summarized in an automatically generated report for skill assessment.
- 8 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8  (8) , 4441-4448
Human-in-the-Loop Embodied Intelligence With Interactive Simulation Environment for Surgical Robot Learning
Surgical robot automation has attracted increasing research interest over the past decade, expecting its potential to benefit surgeons, nurses and patients. Recently, the learning paradigm of embodied intelligence has demonstrated promising ability to learn good control policies for various complex tasks, where embodied AI simulators play an essential role to facilitate relevant research. However, existing open-sourced simulators for surgical robot are still not sufficiently supporting human interactions through physical input devices, which further limits effective investigations on how the human demonstrations would affect policy learning. In this work, we study human-in-the-loop embodied intelligence with a new interactive simulation platform for surgical robot learning. Specifically, we establish our platform based on our previously released SurRoL simulator with several new features co-developed to allow high-quality human interaction via an input device. We showcase the improvement of our simulation environment with the designed new features, and validate effectiveness of incorporating human factors in embodied intelligence through the use of human demonstrations and reinforcement learning as a representative example. Promising results are obtained in terms of learning efficiency. Lastly, five new surgical robot training tasks are developed and released, with which we hope to pave the way for future research on surgical embodied intelligence.
- 8 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8  (8) , 4441-4448
Human-in-the-Loop Embodied Intelligence With Interactive Simulation Environment for Surgical Robot Learning
Surgical robot automation has attracted increasing research interest over the past decade, expecting its potential to benefit surgeons, nurses and patients. Recently, the learning paradigm of embodied intelligence has demonstrated promising ability to learn good control policies for various complex tasks, where embodied AI simulators play an essential role to facilitate relevant research. However, existing open-sourced simulators for surgical robot are still not sufficiently supporting human interactions through physical input devices, which further limits effective investigations on how the human demonstrations would affect policy learning. In this work, we study human-in-the-loop embodied intelligence with a new interactive simulation platform for surgical robot learning. Specifically, we establish our platform based on our previously released SurRoL simulator with several new features co-developed to allow high-quality human interaction via an input device. We showcase the improvement of our simulation environment with the designed new features, and validate effectiveness of incorporating human factors in embodied intelligence through the use of human demonstrations and reinforcement learning as a representative example. Promising results are obtained in terms of learning efficiency. Lastly, five new surgical robot training tasks are developed and released, with which we hope to pave the way for future research on surgical embodied intelligence.