Xiaoyang Zou
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Xiaoyang Zou has a total of 16 co-authors. Here are the top co-authors by number of shared publications:
- Guoyan Zheng
Shanghai Jiao Tong University, Shanghai, China
7 shared publications - Rong Tao
Institute of Medical Robotics, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
2 shared publications - Wenyuan Sun
Institute of Medical Robotics, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
2 shared publications - Baoru Huang
University of Liverpool
1 shared publication - Donghua Hang
Department of Orthopaedics
1 shared publication - Guangcong Ruan
Department of Gastroenterology, Daping Hospital, Army Medical University (Third Military Medical University), Chongqing 400042, China
1 shared publication
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 2510-2518
Enforcing Answer-Location Consistency for Surgical Visual Question Localized-Answering via Reverse Question Learning
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 2510-2518
Open AccessEnforcing Answer-Location Consistency for Surgical Visual Question Localized-Answering via Reverse Question Learning
- 3 June 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 75 (00189456) , 5010816
A Novel Stereo Tracking Technology for Intraoperative Intervertebral Motion Compensation in Robot-Assisted Laminectomy
Robot-Assisted Laminectomy (RAL) has become an emerging approach for treatment of lumbar spinal stenosis. However, the spine is an articulated structure that can be deformed under contact force during grinding. Patient-image registration performed at the start of RAL cannot account for intraoperative intervertebral motion, resulting in degraded surgical accuracy and compromised safety. To tackle this issue, we propose a novel stereo tracking technology for intraoperative intervertebral motion compensation in RAL. Our tracking technology is a marker-based method, consisting of customized markers, a coding system, and a labeling and position tracking framework. Specifically, we first propose a new type of markers featured by binary-coded spatial patterns, referred to as SurgTag marker. Each marker in the codebook has a unique digital IDentification (ID) code and a fixed set of keypoints. By rigidly attaching a marker with a unique ID to each vertebra during RAL, we can simultaneously track multiple vertebrae. We then introduce a two-stage SurgTag-marker-based tracking framework consisting of a marker labeling stage followed by a keypoint detection stage. In theory, we formulate both marker labeling and keypoint detection as one-to-one set prediction problems and further propose respectively two different types of DEtection-TRansformer (DETR)-based detector: Marker-DETR and Keypoint-DETR. Marker poses are then estimated based on detected keypoints, enabling tracking and compensation for intervertebral motion. Comprehensive experiments were conducted for evaluation. The proposed method achieved on average a submillimeter translation error and a sub-degree orientation error on both a synthetic dataset and an ex-vivo dataset. Additionally, surgical robot with the proposed technology achieved a submillimeter grinding error, demonstrating its effectiveness for improving surgical accuracy. The code will be publicly available at https://github.com/Wenyuan-Sun/StereoTracking.
- 3 June 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 75 (00189456) , 5010816
A Novel Stereo Tracking Technology for Intraoperative Intervertebral Motion Compensation in Robot-Assisted Laminectomy
Robot-Assisted Laminectomy (RAL) has become an emerging approach for treatment of lumbar spinal stenosis. However, the spine is an articulated structure that can be deformed under contact force during grinding. Patient-image registration performed at the start of RAL cannot account for intraoperative intervertebral motion, resulting in degraded surgical accuracy and compromised safety. To tackle this issue, we propose a novel stereo tracking technology for intraoperative intervertebral motion compensation in RAL. Our tracking technology is a marker-based method, consisting of customized markers, a coding system, and a labeling and position tracking framework. Specifically, we first propose a new type of markers featured by binary-coded spatial patterns, referred to as SurgTag marker. Each marker in the codebook has a unique digital IDentification (ID) code and a fixed set of keypoints. By rigidly attaching a marker with a unique ID to each vertebra during RAL, we can simultaneously track multiple vertebrae. We then introduce a two-stage SurgTag-marker-based tracking framework consisting of a marker labeling stage followed by a keypoint detection stage. In theory, we formulate both marker labeling and keypoint detection as one-to-one set prediction problems and further propose respectively two different types of DEtection-TRansformer (DETR)-based detector: Marker-DETR and Keypoint-DETR. Marker poses are then estimated based on detected keypoints, enabling tracking and compensation for intervertebral motion. Comprehensive experiments were conducted for evaluation. The proposed method achieved on average a submillimeter translation error and a sub-degree orientation error on both a synthetic dataset and an ex-vivo dataset. Additionally, surgical robot with the proposed technology achieved a submillimeter grinding error, demonstrating its effectiveness for improving surgical accuracy. The code will be publicly available at https://github.com/Wenyuan-Sun/StereoTracking.
- Open Access
- 22 December 2025
- Published by MDPI AG in Bioengineering
- Vol. 13 (1) , 8
Subclass-Aware Contrastive Semi-Supervised Learning for Inflammatory Bowel Disease Classification from Colonoscopy Images
Inflammatory bowel disease (IBD) includes Crohn’s disease (CD) and ulcerative colitis (UC). The accurate classification of IBD from colonoscopy images is critical for diagnosis and treatment. However, the lack of labeled data poses a major challenge for developing deep learning-based IBD classification approaches. Recently, pseudo-labeling-based semi-supervised learning methods offer a promising solution in leveraging both labeled and unlabeled data to improve classification performance. Nevertheless, due to significant intra-class variability and the subtle inter-class differences in IBD colonoscopy images, pseudo-labels are often inaccurate, which results in confirmation bias and suboptimal performance. To address this challenge, a Subclass-Aware Contrastive Semi-Supervised Learning method, referred to as SACSSL, is proposed for accurate IBD classification by integrating a subclass-aware contrastive module into a pseudo-labeling-based semi-supervised framework, e.g., FixMatch. Specifically, unlabeled samples are first partitioned into confident and uncertain samples according to the confidence of pseudo-labels. An instance-level contrastive loss is then applied to uncertain samples, aiming to mitigate confirmation bias. Furthermore, intra-class heterogeneity is captured by introducing a set of prototypes for each subclass and assigning confident samples to these prototypes to form fine-grained subclasses, and supervised contrastive loss is applied to promote intra-subclass clustering, thereby enhancing inter-class separability while preserving intra-class diversity. Our method is evaluated on two datasets, i.e., an in-house collected Daping dataset for IBD classification and a publicly available LIMUC dataset for UC severity grading. On both datasets, our method achieves state-of-the-art performance under the semi-supervised setting. Specifically, with only 20% labeled data, the proposed method reaches an overall accuracy of 93.2% and an F1-score of 80.1% on the Daping dataset, which is close to the fully supervised upper bound (94.0% accuracy and 80.8% F1-score), and it achieves an overall accuracy of 76.4% and an F1-score of 68.9% on the LIMUC dataset. Comprehensive experimental results demonstrate the effectiveness of our method for semi-supervised colonoscopy image classification.
- 22 December 2025
- Published by MDPI AG in Bioengineering
- Vol. 13 (1) , 8
Open AccessSubclass-Aware Contrastive Semi-Supervised Learning for Inflammatory Bowel Disease Classification from Colonoscopy Images
Inflammatory bowel disease (IBD) includes Crohn’s disease (CD) and ulcerative colitis (UC). The accurate classification of IBD from colonoscopy images is critical for diagnosis and treatment. However, the lack of labeled data poses a major challenge for developing deep learning-based IBD classification approaches. Recently, pseudo-labeling-based semi-supervised learning methods offer a promising solution in leveraging both labeled and unlabeled data to improve classification performance. Nevertheless, due to significant intra-class variability and the subtle inter-class differences in IBD colonoscopy images, pseudo-labels are often inaccurate, which results in confirmation bias and suboptimal performance. To address this challenge, a Subclass-Aware Contrastive Semi-Supervised Learning method, referred to as SACSSL, is proposed for accurate IBD classification by integrating a subclass-aware contrastive module into a pseudo-labeling-based semi-supervised framework, e.g., FixMatch. Specifically, unlabeled samples are first partitioned into confident and uncertain samples according to the confidence of pseudo-labels. An instance-level contrastive loss is then applied to uncertain samples, aiming to mitigate confirmation bias. Furthermore, intra-class heterogeneity is captured by introducing a set of prototypes for each subclass and assigning confident samples to these prototypes to form fine-grained subclasses, and supervised contrastive loss is applied to promote intra-subclass clustering, thereby enhancing inter-class separability while preserving intra-class diversity. Our method is evaluated on two datasets, i.e., an in-house collected Daping dataset for IBD classification and a publicly available LIMUC dataset for UC severity grading. On both datasets, our method achieves state-of-the-art performance under the semi-supervised setting. Specifically, with only 20% labeled data, the proposed method reaches an overall accuracy of 93.2% and an F1-score of 80.1% on the Daping dataset, which is close to the fully supervised upper bound (94.0% accuracy and 80.8% F1-score), and it achieves an overall accuracy of 76.4% and an F1-score of 68.9% on the LIMUC dataset. Comprehensive experimental results demonstrate the effectiveness of our method for semi-supervised colonoscopy image classification.
- 10 May 2024
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 20 (1) , 43-55
A fully automatic fiducial detection and correspondence establishing method for online C-arm calibration
Purpose Online C-arm calibration with a mobile fiducial cage plays an essential role in various image-guided interventions. However, it is challenging to develop a fully automatic approach, which requires not only an accurate detection of fiducial projections but also a robust 2D–3D correspondence establishment. Methods We propose a novel approach for online C-arm calibration with a mobile fiducial cage. Specifically, a novel mobile calibration cage embedded with 16 fiducials is designed, where the fiducials are arranged to form 4 line patterns with different cross-ratios. Then, an auto-context-based detection network (ADNet) is proposed to perform an accurate and robust detection of 2D projections of those fiducials in acquired C-arm images. Subsequently, we present a cross-ratio consistency-based 2D–3D correspondence establishing method to automatically match the detected 2D fiducial projections with those 3D fiducials, allowing for an accurate online C-arm calibration. Results We designed and conducted comprehensive experiments to evaluate the proposed approach. For automatic detection of 2D fiducial projections, the proposed ADNet achieved a mean point-to-point distance of \(0.65 \pm 1.33\) pixels. Additionally, the proposed C-arm calibration approach achieved a mean re-projection error of \(1.01 \pm 0.63\) pixels and a mean point-to-line distance of \(0.22 \pm 0.12\) mm. When the proposed C-arm calibration approach was applied to downstream tasks involving landmark and surface model reconstruction, sub-millimeter accuracy was achieved. Conclusion In summary, we developed a novel approach for online C-arm calibration. Both qualitative and quantitative results of comprehensive experiments demonstrated the accuracy and robustness of the proposed approach. Our approach holds potentials for various image-guided interventions.- 10 May 2024
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 20 (1) , 43-55
A fully automatic fiducial detection and correspondence establishing method for online C-arm calibration
Purpose Online C-arm calibration with a mobile fiducial cage plays an essential role in various image-guided interventions. However, it is challenging to develop a fully automatic approach, which requires not only an accurate detection of fiducial projections but also a robust 2D–3D correspondence establishment. Methods We propose a novel approach for online C-arm calibration with a mobile fiducial cage. Specifically, a novel mobile calibration cage embedded with 16 fiducials is designed, where the fiducials are arranged to form 4 line patterns with different cross-ratios. Then, an auto-context-based detection network (ADNet) is proposed to perform an accurate and robust detection of 2D projections of those fiducials in acquired C-arm images. Subsequently, we present a cross-ratio consistency-based 2D–3D correspondence establishing method to automatically match the detected 2D fiducial projections with those 3D fiducials, allowing for an accurate online C-arm calibration. Results We designed and conducted comprehensive experiments to evaluate the proposed approach. For automatic detection of 2D fiducial projections, the proposed ADNet achieved a mean point-to-point distance of \(0.65 \pm 1.33\) pixels. Additionally, the proposed C-arm calibration approach achieved a mean re-projection error of \(1.01 \pm 0.63\) pixels and a mean point-to-line distance of \(0.22 \pm 0.12\) mm. When the proposed C-arm calibration approach was applied to downstream tasks involving landmark and surface model reconstruction, sub-millimeter accuracy was achieved. Conclusion In summary, we developed a novel approach for online C-arm calibration. Both qualitative and quantitative results of comprehensive experiments demonstrated the accuracy and robustness of the proposed approach. Our approach holds potentials for various image-guided interventions.- 3 April 2024
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 19 (9) , 1723-1731
Incremental regression of localization context for automatic segmentation of ossified ligamentum flavum from CT data
Purpose Segmentation of ossified ligamentum flavum (OLF) plays a crucial role in developing computer-assisted, image-guided systems for decompressive thoracic laminectomy. Manual segmentation is time-consuming, tedious, and label-intensive. It also suffers from inter- and intra-observer variability. Automatic segmentation is highly desired. Methods A two-stage, localization context-aware framework is developed for automatic segmentation of ossified ligamentum flavum. In the first stage, localization heatmaps of OLFs are obtained via incremental regression. In the second stage, the obtained heatmaps are then treated as the localization context for a segmentation U-Net. Our framework can directly map a whole volumetic data to its volume-wise labels. Results We designed and conducted comprehensive experiments on datasets of 100 patients to evaluate the performance of the proposed method. Our method achieved an average Dice similarity coefficient of 61.2 ± 7.6%, an average surface distance of 1.1 ± 0.5 mm, and an average positive predictive value of 62.0 ± 12.8%. Conclusion To the best knowledge of the authors, this is the first study aiming for automatic segmentation of ossified ligamentum flavum. Results from the comprehensive experiments demonstrate the superior performance of the proposed method over the state-of-the-art methods.
- 3 April 2024
- Published by Springer Nature in International Journal of Computer Assisted Radiology and Surgery
- Vol. 19 (9) , 1723-1731
Incremental regression of localization context for automatic segmentation of ossified ligamentum flavum from CT data
Purpose Segmentation of ossified ligamentum flavum (OLF) plays a crucial role in developing computer-assisted, image-guided systems for decompressive thoracic laminectomy. Manual segmentation is time-consuming, tedious, and label-intensive. It also suffers from inter- and intra-observer variability. Automatic segmentation is highly desired. Methods A two-stage, localization context-aware framework is developed for automatic segmentation of ossified ligamentum flavum. In the first stage, localization heatmaps of OLFs are obtained via incremental regression. In the second stage, the obtained heatmaps are then treated as the localization context for a segmentation U-Net. Our framework can directly map a whole volumetic data to its volume-wise labels. Results We designed and conducted comprehensive experiments on datasets of 100 patients to evaluate the performance of the proposed method. Our method achieved an average Dice similarity coefficient of 61.2 ± 7.6%, an average surface distance of 1.1 ± 0.5 mm, and an average positive predictive value of 62.0 ± 12.8%. Conclusion To the best knowledge of the authors, this is the first study aiming for automatic segmentation of ossified ligamentum flavum. Results from the comprehensive experiments demonstrate the superior performance of the proposed method over the state-of-the-art methods.
- 25 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 42 (11) , 3256-3268
LAST: LAtent Space-Constrained Transformers for Automatic Surgical Phase Recognition and Tool Presence Detection
When developing context-aware systems, automatic surgical phase recognition and tool presence detection are two essential tasks. There exist previous attempts to develop methods for both tasks but majority of the existing methods utilize a frame-level loss function (e.g., cross-entropy) which does not fully leverage the underlying semantic structure of a surgery, leading to sub-optimal results. In this paper, we propose multi-task learning-based, LAtent Space-constrained Transformers, referred as LAST, for automatic surgical phase recognition and tool presence detection. Our design features a two-branch transformer architecture with a novel and generic way to leverage video-level semantic information during network training. This is done by learning a non-linear compact presentation of the underlying semantic structure information of surgical videos through a transformer variational autoencoder (VAE) and by encouraging models to follow the learned statistical distributions. In other words, LAST is of structure-aware and favors predictions that lie on the extracted low dimensional data manifold. Validated on two public datasets of the cholecystectomy surgery, i.e., the Cholec80 dataset and the M2cai16 dataset, our method achieves better results than other state-of-the-art methods. Specifically, on the Cholec80 dataset, our method achieves an average accuracy of 93.12±4.71%, an average precision of 89.25±5.49%, an average recall of 90.10±5.45% and an average Jaccard of 81.11 ±7.62% for phase recognition, and an average mAP of 95.15±3.87% for tool presence detection. Similar superior performance is also observed when LAST is applied to the M2cai16 dataset.
- 25 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 42 (11) , 3256-3268
LAST: LAtent Space-Constrained Transformers for Automatic Surgical Phase Recognition and Tool Presence Detection
When developing context-aware systems, automatic surgical phase recognition and tool presence detection are two essential tasks. There exist previous attempts to develop methods for both tasks but majority of the existing methods utilize a frame-level loss function (e.g., cross-entropy) which does not fully leverage the underlying semantic structure of a surgery, leading to sub-optimal results. In this paper, we propose multi-task learning-based, LAtent Space-constrained Transformers, referred as LAST, for automatic surgical phase recognition and tool presence detection. Our design features a two-branch transformer architecture with a novel and generic way to leverage video-level semantic information during network training. This is done by learning a non-linear compact presentation of the underlying semantic structure information of surgical videos through a transformer variational autoencoder (VAE) and by encouraging models to follow the learned statistical distributions. In other words, LAST is of structure-aware and favors predictions that lie on the extracted low dimensional data manifold. Validated on two public datasets of the cholecystectomy surgery, i.e., the Cholec80 dataset and the M2cai16 dataset, our method achieves better results than other state-of-the-art methods. Specifically, on the Cholec80 dataset, our method achieves an average accuracy of 93.12±4.71%, an average precision of 89.25±5.49%, an average recall of 90.10±5.45% and an average Jaccard of 81.11 ±7.62% for phase recognition, and an average mAP of 95.15±3.87% for tool presence detection. Similar superior performance is also observed when LAST is applied to the M2cai16 dataset.
- 18 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Robotics and Bionics
- Vol. 5 (1) , 42-53
SVT-SDE: Spatiotemporal Vision Transformers-Based Self-Supervised Depth Estimation in Stereoscopic Surgical Videos
Dense depth estimation plays a crucial role in developing context-aware computer-assisted intervention systems. However, it is a challenging task due to low image quality and highly dynamic surgical environment. The task is further complicated by the difficulty in acquiring per-pixel ground truth depth data in a surgical setting. Recent works on self-supervised depth estimation use image reconstruction (i.e., the warped images) as supervisory signal, which helps to eliminate the requirement of ground truth depth annotations but also causes over-smoothed depth predictions. Additionally, most existing depth estimation methods are built upon static laparoscopic images, ignoring rich temporal information. To address these challenges, we propose a novel spatiotemporal vision transformers-based self-supervised depth estimation method, referred as SVT-SDE. Unlike previous works, SVT-SDE features a novel spatiotemporal vision transformers (SVT) architecture, which can learn complementary visual and temporal information from the input stereoscopic video clips. We further introduce high-frequency-based supervisory signal, which helps to preserve fine-grained details of depth estimation. Results from experiments conducted on two publicly available datasets demonstrate the superior performance of SVT-SDE over the state-of-the-art self-supervised depth estimation methods.
- 18 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Robotics and Bionics
- Vol. 5 (1) , 42-53
SVT-SDE: Spatiotemporal Vision Transformers-Based Self-Supervised Depth Estimation in Stereoscopic Surgical Videos
Dense depth estimation plays a crucial role in developing context-aware computer-assisted intervention systems. However, it is a challenging task due to low image quality and highly dynamic surgical environment. The task is further complicated by the difficulty in acquiring per-pixel ground truth depth data in a surgical setting. Recent works on self-supervised depth estimation use image reconstruction (i.e., the warped images) as supervisory signal, which helps to eliminate the requirement of ground truth depth annotations but also causes over-smoothed depth predictions. Additionally, most existing depth estimation methods are built upon static laparoscopic images, ignoring rich temporal information. To address these challenges, we propose a novel spatiotemporal vision transformers-based self-supervised depth estimation method, referred as SVT-SDE. Unlike previous works, SVT-SDE features a novel spatiotemporal vision transformers (SVT) architecture, which can learn complementary visual and temporal information from the input stereoscopic video clips. We further introduce high-frequency-based supervisory signal, which helps to preserve fine-grained details of depth estimation. Results from experiments conducted on two publicly available datasets demonstrate the superior performance of SVT-SDE over the state-of-the-art self-supervised depth estimation methods.