You Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
You Zhang has a total of 171 co-authors. Here are the top co-authors by number of shared publications:
- Jing Wang
University of Texas Southwestern Medical Center
17 shared publications - Hua-Chieh Shao
University of Texas Southwestern Medical Center
13 shared publications - Yunxiang Li
Department of Radiation Oncology, The University of Texas Southwestern Medical Center, 2280 Inwood Rd, Dallas, Texas, 75390-9096, UNITED STATES
11 shared publications Weiguo Lu
Department of Radiation Oncology The Medical Artificial Intelligence and Automation (MAIA) Laboratory University of Texas Southwestern Medical Center 75390 Dallas TX USA
9 shared publicationsGuoping Xu
Department of Radiation Oncology The Medical Artificial Intelligence and Automation (MAIA) Laboratory University of Texas Southwestern Medical Center 75390 Dallas TX USA
7 shared publicationsHua-Chieh Shao
Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX, United States of America
7 shared publications
Latest Publications
- Open Access
- 9 October 2026
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 71 (19) , 195030
DiffINR: conditional neural field diffusion for deformation-driven volumetric image estimation from limited measurements
Objective. We propose DiffINR for rapid deformation-driven estimation of patient-specific target volumes from limited measurements. DiffINR uses a coordinate-based implicit neural representation (INR) to estimate a dense deformation vector field (DVF) that warps a high-quality prior volume (source) to the target anatomy, with patient-specific INR initialization provided by a population-based conditional hyperdiffusion model. Approach. During offline training, INRs were optimized on fully sampled source-target image volume pairs to represent DVFs. A Transformer diffusion model, hyperdiffusion, learned the distribution of the resulting INR weights. The model was jointly conditioned on anatomical features extracted from the training source and target volumes by a foundation model and on features encoded directly from limited measurements. At inference, hyperdiffusion generated INR initial weights, which were further fine-tuned by patient-specific optimization using measurement-domain fidelity and DVF-smoothness losses. We evaluated DiffINR on two tasks: cone-beam computed tomography (CBCT) estimation from limited-angle x-ray projections and real-time volumetric magnetic resonance imaging (MRI) estimation from ultra-sparse k-space data. Main Results. For orthogonal-view 90° CBCT estimation, DiffINR achieved a structural similarity index measure (SSIM) of 0.962 ± 0.021, relative error of 6.84% ± 1.70%, and target registration error of 3.61 ± 1.27 mm, outperforming the iterative methods, population-based models, and randomly initialized INR baselines. For volumetric MRI estimation with 13 radial spokes per slice, DiffINR achieved an SSIM of 0.985 ± 0.007, Dice coefficient of 0.89 ± 0.04, and center-of-mass error of 1.30 ± 1.09 mm. DiffINR showed high robustness to data distribution shifts at test time, including dataset changes, limited-angle setting variations, and different MRI sampling sparsity levels. Test-time runtime was 23 s for CBCT estimation and 27 s for MRI estimation, compared with 795 s and 928 s, respectively, for INR optimization from scratch. Significance. By combining hyperdiffusion-based INR initialization with patient-specific data consistency, DiffINR achieves fast and accurate deformation-driven volumetric image estimation.
- 9 October 2026
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 71 (19) , 195030
Open AccessDiffINR: conditional neural field diffusion for deformation-driven volumetric image estimation from limited measurements
Objective. We propose DiffINR for rapid deformation-driven estimation of patient-specific target volumes from limited measurements. DiffINR uses a coordinate-based implicit neural representation (INR) to estimate a dense deformation vector field (DVF) that warps a high-quality prior volume (source) to the target anatomy, with patient-specific INR initialization provided by a population-based conditional hyperdiffusion model. Approach. During offline training, INRs were optimized on fully sampled source-target image volume pairs to represent DVFs. A Transformer diffusion model, hyperdiffusion, learned the distribution of the resulting INR weights. The model was jointly conditioned on anatomical features extracted from the training source and target volumes by a foundation model and on features encoded directly from limited measurements. At inference, hyperdiffusion generated INR initial weights, which were further fine-tuned by patient-specific optimization using measurement-domain fidelity and DVF-smoothness losses. We evaluated DiffINR on two tasks: cone-beam computed tomography (CBCT) estimation from limited-angle x-ray projections and real-time volumetric magnetic resonance imaging (MRI) estimation from ultra-sparse k-space data. Main Results. For orthogonal-view 90° CBCT estimation, DiffINR achieved a structural similarity index measure (SSIM) of 0.962 ± 0.021, relative error of 6.84% ± 1.70%, and target registration error of 3.61 ± 1.27 mm, outperforming the iterative methods, population-based models, and randomly initialized INR baselines. For volumetric MRI estimation with 13 radial spokes per slice, DiffINR achieved an SSIM of 0.985 ± 0.007, Dice coefficient of 0.89 ± 0.04, and center-of-mass error of 1.30 ± 1.09 mm. DiffINR showed high robustness to data distribution shifts at test time, including dataset changes, limited-angle setting variations, and different MRI sampling sparsity levels. Test-time runtime was 23 s for CBCT estimation and 27 s for MRI estimation, compared with 795 s and 928 s, respectively, for INR optimization from scratch. Significance. By combining hyperdiffusion-based INR initialization with patient-specific data consistency, DiffINR achieves fast and accurate deformation-driven volumetric image estimation.
- Open Access
- 3 September 2026
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 71 (17) , 175018
S2V-DREME: a time-resolved slice-to-volume MR image reconstruction framework with dynamic reconstruction and motion estimation
Objective. Existing volumetric magnetic resonance imaging (MRI) techniques are constrained by the trade-off between acquisition time and image quality, limiting accuracy in motion-impacted sites such as the liver. To enable fast, better-quality volumetric imaging with sufficient spatiotemporal resolution, we developed a time-resolved volumetric MRI technique that recovers 3D volumes from acquired 2D MR slices for real-time 3D anatomy and motion tracking. Approach. 2D MR slices dynamically acquired in time and space were mapped to time-resolved 3D MRIs using a one-shot slice-to-volume framework, S2V-DREME. The model jointly estimates a reference 3D MRI and time-resolved deformation vector fields (DVFs) that warp the reference volume into dynamic 3D MRIs. The reference volume is represented by a spatial implicit neural representation (INR), while the DVFs are derived via low-rank motion modeling. Motion basis components (MBCs) are generated by a spline-enhanced INR (SINR)-based motion generator, with coefficients inferred by a feature-wise linear modulation-based motion encoder. A progressive optimization strategy sequentially initializes the spatial INR and MBCs before joint optimization. The loss function integrates slice data fidelity, total variation regularization, MBC normalization, and DVF smoothness constraints. Main results. S2V-DREME generates time-resolved volumetric MRIs from 2D MR slice inputs. It was evaluated on digital phantom extended cardiac torso (XCAT), physical phantom, and human studies. In XCAT, it accurately captured regular and irregular motion during dynamic reconstruction (training stage, Dice similarity coefficient (DSC)/COME: 0.92 ± 0.03/0.98 ± 0.43 mm) and real-time motion estimation (testing stage, DSC/COME: 0.91 ± 0.02/0.99 ± 0.73 mm). Physical phantom experiments achieved a mean COME of 1.16 mm, and human studies demonstrated the feasibility of time-resolved 3D reconstruction from orthogonal-view and single-view slice acquisitions. Significance. By combining a novel step-and-shoot acquisition protocol with motion-compensated one-shot learning, S2V-DREME enables accurate time-resolved volumetric MRI reconstruction and motion tracking from cineslices, with strong potential for rapid volumetric imaging and real-time MR-guided adaptive radiotherapy.
- 3 September 2026
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 71 (17) , 175018
Open AccessS2V-DREME: a time-resolved slice-to-volume MR image reconstruction framework with dynamic reconstruction and motion estimation
Objective. Existing volumetric magnetic resonance imaging (MRI) techniques are constrained by the trade-off between acquisition time and image quality, limiting accuracy in motion-impacted sites such as the liver. To enable fast, better-quality volumetric imaging with sufficient spatiotemporal resolution, we developed a time-resolved volumetric MRI technique that recovers 3D volumes from acquired 2D MR slices for real-time 3D anatomy and motion tracking. Approach. 2D MR slices dynamically acquired in time and space were mapped to time-resolved 3D MRIs using a one-shot slice-to-volume framework, S2V-DREME. The model jointly estimates a reference 3D MRI and time-resolved deformation vector fields (DVFs) that warp the reference volume into dynamic 3D MRIs. The reference volume is represented by a spatial implicit neural representation (INR), while the DVFs are derived via low-rank motion modeling. Motion basis components (MBCs) are generated by a spline-enhanced INR (SINR)-based motion generator, with coefficients inferred by a feature-wise linear modulation-based motion encoder. A progressive optimization strategy sequentially initializes the spatial INR and MBCs before joint optimization. The loss function integrates slice data fidelity, total variation regularization, MBC normalization, and DVF smoothness constraints. Main results. S2V-DREME generates time-resolved volumetric MRIs from 2D MR slice inputs. It was evaluated on digital phantom extended cardiac torso (XCAT), physical phantom, and human studies. In XCAT, it accurately captured regular and irregular motion during dynamic reconstruction (training stage, Dice similarity coefficient (DSC)/COME: 0.92 ± 0.03/0.98 ± 0.43 mm) and real-time motion estimation (testing stage, DSC/COME: 0.91 ± 0.02/0.99 ± 0.73 mm). Physical phantom experiments achieved a mean COME of 1.16 mm, and human studies demonstrated the feasibility of time-resolved 3D reconstruction from orthogonal-view and single-view slice acquisitions. Significance. By combining a novel step-and-shoot acquisition protocol with motion-compensated one-shot learning, S2V-DREME enables accurate time-resolved volumetric MRI reconstruction and motion tracking from cineslices, with strong potential for rapid volumetric imaging and real-time MR-guided adaptive radiotherapy.
- Open Access
- 1 June 2026
- Published by IOP Publishing in Machine Learning: Science and Technology
- Vol. 7 (3) , 035050
Exploiting DINOv3-based self-supervised features for robust few-shot medical image segmentation
Deep learning-based automatic medical image segmentation plays a critical role in clinical diagnosis and treatment planning but remains challenging in few-shot scenarios due to the scarcity of annotated training data. Recently, self-supervised foundation models such as DINOv3, which were trained on large natural image datasets, have shown strong potential for dense feature extraction that can help with the few-shot learning challenge. Yet, their direct application to medical images is hindered by domain differences. In this work, we propose DINO-AugSeg, a novel framework that leverages DINOv3 features to address the few-shot medical image segmentation challenge. Specifically, we introduce WT-Aug, a wavelet-based feature-level augmentation module that enriches the diversity of DINOv3-extracted features by perturbing frequency components, and CG-Fuse, a contextual information-guided fusion module that exploits cross-attention to integrate semantic-rich low-resolution features with spatially detailed high-resolution features. Extensive experiments on six public benchmarks spanning five imaging modalities, including MRI, CT, ultrasound, endoscopy, and dermoscopy, demonstrate that DINO-AugSeg consistently outperforms existing methods under limited-sample conditions. The results highlight the effectiveness of incorporating wavelet-domain augmentation and contextual fusion for robust feature representation, suggesting DINO-AugSeg as a promising direction for advancing few-shot medical image segmentation. Code and data will be made available on https://github.com/apple1986/DINO-AugSeg .
- 1 June 2026
- Published by IOP Publishing in Machine Learning: Science and Technology
- Vol. 7 (3) , 035050
Open AccessExploiting DINOv3-based self-supervised features for robust few-shot medical image segmentation
Deep learning-based automatic medical image segmentation plays a critical role in clinical diagnosis and treatment planning but remains challenging in few-shot scenarios due to the scarcity of annotated training data. Recently, self-supervised foundation models such as DINOv3, which were trained on large natural image datasets, have shown strong potential for dense feature extraction that can help with the few-shot learning challenge. Yet, their direct application to medical images is hindered by domain differences. In this work, we propose DINO-AugSeg, a novel framework that leverages DINOv3 features to address the few-shot medical image segmentation challenge. Specifically, we introduce WT-Aug, a wavelet-based feature-level augmentation module that enriches the diversity of DINOv3-extracted features by perturbing frequency components, and CG-Fuse, a contextual information-guided fusion module that exploits cross-attention to integrate semantic-rich low-resolution features with spatially detailed high-resolution features. Extensive experiments on six public benchmarks spanning five imaging modalities, including MRI, CT, ultrasound, endoscopy, and dermoscopy, demonstrate that DINO-AugSeg consistently outperforms existing methods under limited-sample conditions. The results highlight the effectiveness of incorporating wavelet-domain augmentation and contextual fusion for robust feature representation, suggesting DINO-AugSeg as a promising direction for advancing few-shot medical image segmentation. Code and data will be made available on https://github.com/apple1986/DINO-AugSeg .
- Open Access
- 30 April 2026
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 71 (9)
Mid-range arc therapy for efficient and RBE-robust proton treatment
Objective. Proton arc therapy (PAT) has the potential to improve plan conformity while enhancing normal tissue sparing; however, its clinical translation faces practical barriers related to radiobiological uncertainties and delivery inefficiency. This study introduces a novel planning approach, Mid-Range Proton Arc Therapy (MRPAT), designed to enhance delivery efficiency and mitigate range-related uncertainties in PAT. Approach. MRPAT employs a single mid-range energy layer (MREL) per beam direction, positioning the Bragg peak near the target center (mid-range) plane to confine range uncertainty within the target while reducing dose-averaged linear energy transfer (LETd) and relative biological effectiveness (RBE) hotspots in adjacent organs at risk (OARs). The feasibility of MRPAT was investigated on an ellipse phantom and three clinical cases (prostate, spine, and head and neck). Physical dose distributions, LETd distributions, RBE-weighted dose distributions, and delivery efficiency were compared with full-range arc employing all possible energy layers (Full-Arc) and intensity-modulated proton therapy (IMPT) plans with two beams. Main results. Compared with IMPT, arc plans provided substantial improvements in entrance dose control and OAR sparing. The MRPAT plan achieved comparable target coverage and OAR sparing compared to the baseline Full-Arc plan, demonstrating the redundancy of utilizing all possible energy layers. MRPAT effectively confined LETd and RBE hotspots to the center of the target, reducing potential biological weighted dose spillage over the surroundings. In terms of delivery efficiency, MRPAT eliminates energy layer switching within the same control point by using less than 5% of the energy layers, requiring only about 20% of the total beam delivery time compared with the Full-Arc plan. Significance. MRPAT demonstrates a simple, efficient, and high‑LETd/RBE hotspot‑containment planning concept that is practical for clinical use and adaptable to future adaptive PAT workflows.
- 30 April 2026
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 71 (9)
Open AccessMid-range arc therapy for efficient and RBE-robust proton treatment
Objective. Proton arc therapy (PAT) has the potential to improve plan conformity while enhancing normal tissue sparing; however, its clinical translation faces practical barriers related to radiobiological uncertainties and delivery inefficiency. This study introduces a novel planning approach, Mid-Range Proton Arc Therapy (MRPAT), designed to enhance delivery efficiency and mitigate range-related uncertainties in PAT. Approach. MRPAT employs a single mid-range energy layer (MREL) per beam direction, positioning the Bragg peak near the target center (mid-range) plane to confine range uncertainty within the target while reducing dose-averaged linear energy transfer (LETd) and relative biological effectiveness (RBE) hotspots in adjacent organs at risk (OARs). The feasibility of MRPAT was investigated on an ellipse phantom and three clinical cases (prostate, spine, and head and neck). Physical dose distributions, LETd distributions, RBE-weighted dose distributions, and delivery efficiency were compared with full-range arc employing all possible energy layers (Full-Arc) and intensity-modulated proton therapy (IMPT) plans with two beams. Main results. Compared with IMPT, arc plans provided substantial improvements in entrance dose control and OAR sparing. The MRPAT plan achieved comparable target coverage and OAR sparing compared to the baseline Full-Arc plan, demonstrating the redundancy of utilizing all possible energy layers. MRPAT effectively confined LETd and RBE hotspots to the center of the target, reducing potential biological weighted dose spillage over the surroundings. In terms of delivery efficiency, MRPAT eliminates energy layer switching within the same control point by using less than 5% of the energy layers, requiring only about 20% of the total beam delivery time compared with the Full-Arc plan. Significance. MRPAT demonstrates a simple, efficient, and high‑LETd/RBE hotspot‑containment planning concept that is practical for clinical use and adaptable to future adaptive PAT workflows.
- Open Access
- 3 February 2026
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 71 (4)
Landmark matching and B-spline implicit neural representations for diffusion-weighted imaging distortion correction
Objective: Geometric distortions in diffusion-weighted imaging (DWI) compromise accurate tumor delineation and spatial localization, limiting its utility in radiation therapy planning and response monitoring. These distortions can be corrected through multimodal registration between distorted DWI and undistorted anatomical images, while conventional mutual information-based optimization often fails due to local minima and produces non-smooth, physically implausible deformations. Approach: This study proposes a landmark matching B-spline implicit neural representation (LMBS-INR) framework for DWI distortion correction. The method integrates anatomical correspondences from a foundation landmark matching model with B-spline parameterized deformation fields to overcome local minima inherent in mutual information optimization. The framework employs Fourier-encoded multi-layer perceptrons to model B-spline deformation fields while ensuring physically plausible transformations, enabling robust multimodal registration between distorted DWI and anatomical references. Main Results: Evaluation on brain and abdominal datasets demonstrated superior performance compared to established methods. The proposed approach achieved average Dice coefficients of 0.919 ± 0.038 (brain) and 0.926 ± 0.032 (abdomen), significantly outperforming all baseline methods. On simulated data, our method achieved an average PSNR of 25.912 ± 3.148 dB, NCC of 0.911 ± 0.137, and SSIM of 0.888 ± 0.107, the best among all methods. Significance: By combining the regularization properties of B-spline parameterization with the cross-modal matching capabilities of foundation models, our method achieves more accurate correction of geometric distortions in DWI, with the potential to enhance the precision of intra/post-radiotherapy assessment.
- 3 February 2026
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 71 (4)
Open AccessLandmark matching and B-spline implicit neural representations for diffusion-weighted imaging distortion correction
Objective: Geometric distortions in diffusion-weighted imaging (DWI) compromise accurate tumor delineation and spatial localization, limiting its utility in radiation therapy planning and response monitoring. These distortions can be corrected through multimodal registration between distorted DWI and undistorted anatomical images, while conventional mutual information-based optimization often fails due to local minima and produces non-smooth, physically implausible deformations. Approach: This study proposes a landmark matching B-spline implicit neural representation (LMBS-INR) framework for DWI distortion correction. The method integrates anatomical correspondences from a foundation landmark matching model with B-spline parameterized deformation fields to overcome local minima inherent in mutual information optimization. The framework employs Fourier-encoded multi-layer perceptrons to model B-spline deformation fields while ensuring physically plausible transformations, enabling robust multimodal registration between distorted DWI and anatomical references. Main Results: Evaluation on brain and abdominal datasets demonstrated superior performance compared to established methods. The proposed approach achieved average Dice coefficients of 0.919 ± 0.038 (brain) and 0.926 ± 0.032 (abdomen), significantly outperforming all baseline methods. On simulated data, our method achieved an average PSNR of 25.912 ± 3.148 dB, NCC of 0.911 ± 0.137, and SSIM of 0.888 ± 0.107, the best among all methods. Significance: By combining the regularization properties of B-spline parameterization with the cross-modal matching capabilities of foundation models, our method achieves more accurate correction of geometric distortions in DWI, with the potential to enhance the precision of intra/post-radiotherapy assessment.
- Open Access
- 19 January 2026
- Published by IOP Publishing in Machine Learning: Health
- Vol. 2 (1) , 015001
Artificial intelligence (AI)-based multi-organ contour quality assurance with uncertainty estimation for online adaptive radiotherapy (oART)
Accurate delineation of treatment targets and organs at risk (OARs) is essential to the success of radiotherapy (RT). Although artificial intelligence (AI)–based segmentation methods have successfully automated the delineation process, a reliable and efficient quality assurance (QA) mechanism is still missing, particularly in the time-critical setting of online adaptive RT (oART). This study aims to address this unmet clinical need by introducing an AI-driven multi-organ contour QA framework with uncertainty quantification. Using MR-guided oART for prostate cancer as the testbed, we developed an automatic multi-structure QA framework by employing a carefully designed contour quality estimation (ConQuE) model. Built on a ResNet34 backbone, ConQuE processes binary segmentation masks with corresponding MR images to classify contours as ‘acceptable’ or ‘revision required’ in a slice-by-slice manner. To ensure scalability, we proposed a unified framework for multi-organ QA by embedding structure-specific code into the fully connected layers. To quantify classification uncertainty, Monte Carlo (MC) dropout was employed, enabling confidence assessment of model predictions for informative decision-making in oART. ConQuE was trained to assess the segmentation quality of prostate and six OARs, including rectum, bladder, urethra, penile bulb, cauda equina, and femoral heads. Training was performed using 249 cases for training, 31 for validation, and another 31 cases were saved for testing. With 30 MC dropout passes, ConQuE can complete QA for one image slice in ∼13.1 ms. Overall, ConQuE achieved accuracy of 93.9% on all structures with structure-specific performance consistently above 90%, demonstrating its strong capability in identifying revision required contours. Moreover, incorrect predictions were strongly correlated with high uncertainty scores, underscoring the effectiveness in improving QA reliability. The proposed AI-based framework holds the potential to automate contour QA with quantified uncertainty. It is well-suited for oART with heavy time constraints given its high accuracy, efficiency, and reliability.
- 19 January 2026
- Published by IOP Publishing in Machine Learning: Health
- Vol. 2 (1) , 015001
Open AccessArtificial intelligence (AI)-based multi-organ contour quality assurance with uncertainty estimation for online adaptive radiotherapy (oART)
Accurate delineation of treatment targets and organs at risk (OARs) is essential to the success of radiotherapy (RT). Although artificial intelligence (AI)–based segmentation methods have successfully automated the delineation process, a reliable and efficient quality assurance (QA) mechanism is still missing, particularly in the time-critical setting of online adaptive RT (oART). This study aims to address this unmet clinical need by introducing an AI-driven multi-organ contour QA framework with uncertainty quantification. Using MR-guided oART for prostate cancer as the testbed, we developed an automatic multi-structure QA framework by employing a carefully designed contour quality estimation (ConQuE) model. Built on a ResNet34 backbone, ConQuE processes binary segmentation masks with corresponding MR images to classify contours as ‘acceptable’ or ‘revision required’ in a slice-by-slice manner. To ensure scalability, we proposed a unified framework for multi-organ QA by embedding structure-specific code into the fully connected layers. To quantify classification uncertainty, Monte Carlo (MC) dropout was employed, enabling confidence assessment of model predictions for informative decision-making in oART. ConQuE was trained to assess the segmentation quality of prostate and six OARs, including rectum, bladder, urethra, penile bulb, cauda equina, and femoral heads. Training was performed using 249 cases for training, 31 for validation, and another 31 cases were saved for testing. With 30 MC dropout passes, ConQuE can complete QA for one image slice in ∼13.1 ms. Overall, ConQuE achieved accuracy of 93.9% on all structures with structure-specific performance consistently above 90%, demonstrating its strong capability in identifying revision required contours. Moreover, incorrect predictions were strongly correlated with high uncertainty scores, underscoring the effectiveness in improving QA reliability. The proposed AI-based framework holds the potential to automate contour QA with quantified uncertainty. It is well-suited for oART with heavy time constraints given its high accuracy, efficiency, and reliability.
- Open Access
- 19 January 2026
- Published by IOP Publishing in Machine Learning: Health
- Vol. 2 (1) , 015003
TSMS-SAM2: multi-scale temporal sampling augmentation and memory-splitting pruning for promptable video object segmentation and tracking in surgical scenarios
Promptable video object segmentation and tracking (VOST) has seen significant advances with the emergence of foundation models like Segment Anything Model 2 (SAM2); however, their application in surgical video analysis remains challenging due to complex motion dynamics and the redundancy of memory that impedes effective learning. In this work, we propose TSMS-SAM2, a novel framework that enhances promptable VOST in surgical videos by addressing challenges of rapid object motion and memory redundancy in SAM2. TSMS-SAM2 introduces two key strategies: multi-temporal-scale video sampling augmentation to improve robustness against motion variability, and a memory splitting and pruning mechanism that organizes and filters past frame features for more efficient and accurate segmentation. Evaluated on EndoVis2017 and EndoVis2018 datasets, TSMS-SAM2 achieved the highest mean (± s.d.) Dice scores of 95.24 ± 0.96% and 86.73 ± 15.46%, respectively, outperforming prior SAM-based and task-specific methods. Extensive ablation studies confirm the effectiveness of multiscale temporal augmentation and memory splitting, highlighting the framework’s potential for robust, efficient segmentation in complex surgical scenarios. Our source code will be made available at https://github.com/apple1986/TSMS-SAM2.
- 19 January 2026
- Published by IOP Publishing in Machine Learning: Health
- Vol. 2 (1) , 015003
Open AccessTSMS-SAM2: multi-scale temporal sampling augmentation and memory-splitting pruning for promptable video object segmentation and tracking in surgical scenarios
Promptable video object segmentation and tracking (VOST) has seen significant advances with the emergence of foundation models like Segment Anything Model 2 (SAM2); however, their application in surgical video analysis remains challenging due to complex motion dynamics and the redundancy of memory that impedes effective learning. In this work, we propose TSMS-SAM2, a novel framework that enhances promptable VOST in surgical videos by addressing challenges of rapid object motion and memory redundancy in SAM2. TSMS-SAM2 introduces two key strategies: multi-temporal-scale video sampling augmentation to improve robustness against motion variability, and a memory splitting and pruning mechanism that organizes and filters past frame features for more efficient and accurate segmentation. Evaluated on EndoVis2017 and EndoVis2018 datasets, TSMS-SAM2 achieved the highest mean (± s.d.) Dice scores of 95.24 ± 0.96% and 86.73 ± 15.46%, respectively, outperforming prior SAM-based and task-specific methods. Extensive ablation studies confirm the effectiveness of multiscale temporal augmentation and memory splitting, highlighting the framework’s potential for robust, efficient segmentation in complex surgical scenarios. Our source code will be made available at https://github.com/apple1986/TSMS-SAM2.
- Open Access
Exploiting DINOv3-Based Self-Supervised Features for Robust Few-Shot Medical Image Segmentation
Deep learning-based automatic medical image segmentation plays a critical role in clinical diagnosis and treatment planning but remains challenging in few-shot scenarios due to the scarcity of annotated training data. Recently, self-supervised foundation models such as DINOv3, which were trained on large natural image datasets, have shown strong potential for dense feature extraction that can help with the few-shot learning challenge. Yet, their direct application to medical images is hindered by domain differences. In this work, we propose DINO-AugSeg, a novel framework that leverages DINOv3 features to address the few-shot medical image segmentation challenge. Specifically, we introduce WT-Aug, a wavelet-based feature-level augmentation module that enriches the diversity of DINOv3-extracted features by perturbing frequency components, and CG-Fuse, a contextual information-guided fusion module that exploits cross-attention to integrate semantic-rich low-resolution features with spatially detailed high-resolution features. Extensive experiments on six public benchmarks spanning five imaging modalities, including MRI, CT, ultrasound, endoscopy, and dermoscopy, demonstrate that DINO-AugSeg consistently outperforms existing methods under limited-sample conditions. The results highlight the effectiveness of incorporating wavelet-domain augmentation and contextual fusion for robust feature representation, suggesting DINO-AugSeg as a promising direction for advancing few-shot medical image segmentation. Code and data will be made available on https://github.com/apple1986/DINO-AugSeg.
Open AccessExploiting DINOv3-Based Self-Supervised Features for Robust Few-Shot Medical Image Segmentation
Deep learning-based automatic medical image segmentation plays a critical role in clinical diagnosis and treatment planning but remains challenging in few-shot scenarios due to the scarcity of annotated training data. Recently, self-supervised foundation models such as DINOv3, which were trained on large natural image datasets, have shown strong potential for dense feature extraction that can help with the few-shot learning challenge. Yet, their direct application to medical images is hindered by domain differences. In this work, we propose DINO-AugSeg, a novel framework that leverages DINOv3 features to address the few-shot medical image segmentation challenge. Specifically, we introduce WT-Aug, a wavelet-based feature-level augmentation module that enriches the diversity of DINOv3-extracted features by perturbing frequency components, and CG-Fuse, a contextual information-guided fusion module that exploits cross-attention to integrate semantic-rich low-resolution features with spatially detailed high-resolution features. Extensive experiments on six public benchmarks spanning five imaging modalities, including MRI, CT, ultrasound, endoscopy, and dermoscopy, demonstrate that DINO-AugSeg consistently outperforms existing methods under limited-sample conditions. The results highlight the effectiveness of incorporating wavelet-domain augmentation and contextual fusion for robust feature representation, suggesting DINO-AugSeg as a promising direction for advancing few-shot medical image segmentation. Code and data will be made available on https://github.com/apple1986/DINO-AugSeg.
- Open Access
- 8 January 2026
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 71 (1) , 015033
Band-limited implicit neural representations for diffusion-weighted imaging denoising
Purpose. Diffusion-weighted imaging (DWI) has significant value in disease diagnosis and treatment response monitoring, but its inherent low signal-to-noise ratio (SNR) severely affects image quality and quantification accuracy. Existing denoising techniques often blur important tissue boundary information when suppressing noise. Methods. This study proposes a band-limited implicit neural representation (BL-INR) framework for DWI denoising. The method introduces BL positional encoding based on the frequency response characteristics of the sinc function to restrict INR models from learning high-frequency noise while maintaining strong signal representation capabilities. Furthermore, multi- b -value DWI and structural MRI from the same patient are integrated as anatomical priors, exploiting the correlation between true signals and the statistical independence of noise to achieve effective denoising. Main Results. In clinical DWI data evaluation across four anatomical regions (brain, head and neck, abdomen, and pelvis), BL-INR’s visualization results were superior to existing methods. Under extremely low SNR conditions (SNR = 1) in simulated noise experiments, BL-INR achieved a peak SNR of 35.44 and structural similarity index of 0.933, significantly outperforming other methods. Phantom denoising results showed that BL-INR achieved an average apparent diffusion coefficient value error of only 4.57 × 10 − 5 mm 2 s −1 , the smallest among all methods. Significance. BL-INR provides a novel approach for DWI denoising by limiting the frequency of INR input positional encoding. Its self-supervised learning characteristics require no paired training data and allow convenient clinical application. The method enables the derivation of accurate diffusion parameters, providing a reliable foundation for DWI-based quantitative analysis with significant clinical application value.
- 8 January 2026
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 71 (1) , 015033
Open AccessBand-limited implicit neural representations for diffusion-weighted imaging denoising
Purpose. Diffusion-weighted imaging (DWI) has significant value in disease diagnosis and treatment response monitoring, but its inherent low signal-to-noise ratio (SNR) severely affects image quality and quantification accuracy. Existing denoising techniques often blur important tissue boundary information when suppressing noise. Methods. This study proposes a band-limited implicit neural representation (BL-INR) framework for DWI denoising. The method introduces BL positional encoding based on the frequency response characteristics of the sinc function to restrict INR models from learning high-frequency noise while maintaining strong signal representation capabilities. Furthermore, multi- b -value DWI and structural MRI from the same patient are integrated as anatomical priors, exploiting the correlation between true signals and the statistical independence of noise to achieve effective denoising. Main Results. In clinical DWI data evaluation across four anatomical regions (brain, head and neck, abdomen, and pelvis), BL-INR’s visualization results were superior to existing methods. Under extremely low SNR conditions (SNR = 1) in simulated noise experiments, BL-INR achieved a peak SNR of 35.44 and structural similarity index of 0.933, significantly outperforming other methods. Phantom denoising results showed that BL-INR achieved an average apparent diffusion coefficient value error of only 4.57 × 10 − 5 mm 2 s −1 , the smallest among all methods. Significance. BL-INR provides a novel approach for DWI denoising by limiting the frequency of INR input positional encoding. Its self-supervised learning characteristics require no paired training data and allow convenient clinical application. The method enables the derivation of accurate diffusion parameters, providing a reliable foundation for DWI-based quantitative analysis with significant clinical application value.
- Open Access
- 5 January 2026
- Published by IOP Publishing in Machine Learning: Health
- Vol. 2 (1)
“See” through the surface: Surface-derived Three-dimensional AI-driven Real-time Imaging Solution for Intra-Treatment Image Guidance
Respiratory motion is a long-standing challenge for lung stereotactic body radiotherapy (SBRT), particularly for centrally located lung tumors where increased toxicity demands more precise motion management during treatment. Current two-dimensional imaging approaches are insufficient for 3D tumor deformable motion tracking. In this study, we developed and evaluated a Surface-derived Three-dimensional (3D) AI-driven Real-time (STAR) imaging system by transforming a surface imaging system into a 3D real-time imaging solution. STAR integrated two key components: (1) prior-model-free spatiotemporal implicit neural representation (PMF-STINR): a machine-learning sub-system for pretreatment dynamic cone-beam CT (CBCT) reconstruction and motion modeling; and (2) surface-to-deformation network (Surf2DefNet): a deep-learning model that correlates intra-treatment body surface images with internal 3D anatomy and motion fields, trained based on the dynamic CBCT and motion model output of PMF-STINR. Specifically, PMF-STINR reconstructs a reference CBCT and solves an eigenvector-based motion model from a pretreatment CBCT, while Surf2DefNet predicts the motion eigen-vector weightings from surface images, enabling it to infer real-time CBCTs and motion vector fields (MVFs) using intra-treatment surface maps acquired later. We evaluated STAR imaging using both a digital extended cardiac torso (XCAT) phantom with regular and irregular motion patterns and ten patient datasets. The relative error (RE), center-of-mass error (COME), 95th percentile Hausdorff distance (HD95), Dice coefficients (DICE), and Pearson correlation coefficient (PCC) metrics between STAR images and the ‘GT’ were evaluated. For the XCAT phantom and the ten patients, the mean COME values are within 1 mm for all but one patient (1.3 mm). The RE values were consistently low, and the DICE and PCC values exceeded 0.89 in all cases. The HD95 are all within 2 mm except for one patient (2. 78 mm). These results demonstrate that the STAR imaging system can achieve accurate spatiotemporal reconstructions from surface images, providing CBCTs and MVFs for intra-treatment real-time image-guidance, and has the potential to improve safety and efficacy of SBRT for lung cancer.
- 5 January 2026
- Published by IOP Publishing in Machine Learning: Health
- Vol. 2 (1)
Open Access“See” through the surface: Surface-derived Three-dimensional AI-driven Real-time Imaging Solution for Intra-Treatment Image Guidance
Respiratory motion is a long-standing challenge for lung stereotactic body radiotherapy (SBRT), particularly for centrally located lung tumors where increased toxicity demands more precise motion management during treatment. Current two-dimensional imaging approaches are insufficient for 3D tumor deformable motion tracking. In this study, we developed and evaluated a Surface-derived Three-dimensional (3D) AI-driven Real-time (STAR) imaging system by transforming a surface imaging system into a 3D real-time imaging solution. STAR integrated two key components: (1) prior-model-free spatiotemporal implicit neural representation (PMF-STINR): a machine-learning sub-system for pretreatment dynamic cone-beam CT (CBCT) reconstruction and motion modeling; and (2) surface-to-deformation network (Surf2DefNet): a deep-learning model that correlates intra-treatment body surface images with internal 3D anatomy and motion fields, trained based on the dynamic CBCT and motion model output of PMF-STINR. Specifically, PMF-STINR reconstructs a reference CBCT and solves an eigenvector-based motion model from a pretreatment CBCT, while Surf2DefNet predicts the motion eigen-vector weightings from surface images, enabling it to infer real-time CBCTs and motion vector fields (MVFs) using intra-treatment surface maps acquired later. We evaluated STAR imaging using both a digital extended cardiac torso (XCAT) phantom with regular and irregular motion patterns and ten patient datasets. The relative error (RE), center-of-mass error (COME), 95th percentile Hausdorff distance (HD95), Dice coefficients (DICE), and Pearson correlation coefficient (PCC) metrics between STAR images and the ‘GT’ were evaluated. For the XCAT phantom and the ten patients, the mean COME values are within 1 mm for all but one patient (1.3 mm). The RE values were consistently low, and the DICE and PCC values exceeded 0.89 in all cases. The HD95 are all within 2 mm except for one patient (2. 78 mm). These results demonstrate that the STAR imaging system can achieve accurate spatiotemporal reconstructions from surface images, providing CBCTs and MVFs for intra-treatment real-time image-guidance, and has the potential to improve safety and efficacy of SBRT for lung cancer.