Hua-Chieh Shao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Hua-Chieh Shao has a total of 25 co-authors. Here are the top co-authors by number of shared publications:
- You Zhang
University of Texas Southwestern Medical Center
13 shared publications - Xiaoxue Qian
University of Texas Southwestern Medical Center
5 shared publications - Yunxiang Li
Department of Radiation Oncology, The University of Texas Southwestern Medical Center, 2280 Inwood Rd, Dallas, Texas, 75390-9096, UNITED STATES
4 shared publications - Jiacheng Xie
Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA
3 shared publications - Tielige Mengke
University of Texas Southwestern Medical Center
3 shared publications - Jie Deng
Department of Radiation Oncology, The University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, Texas, 75390-9303, UNITED STATES
2 shared publications
Latest Publications
- Open Access
- 9 October 2026
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 71  (19) , 195030
DiffINR: conditional neural field diffusion for deformation-driven volumetric image estimation from limited measurements
Objective. We propose DiffINR for rapid deformation-driven estimation of patient-specific target volumes from limited measurements. DiffINR uses a coordinate-based implicit neural representation (INR) to estimate a dense deformation vector field (DVF) that warps a high-quality prior volume (source) to the target anatomy, with patient-specific INR initialization provided by a population-based conditional hyperdiffusion model. Approach. During offline training, INRs were optimized on fully sampled source-target image volume pairs to represent DVFs. A Transformer diffusion model, hyperdiffusion, learned the distribution of the resulting INR weights. The model was jointly conditioned on anatomical features extracted from the training source and target volumes by a foundation model and on features encoded directly from limited measurements. At inference, hyperdiffusion generated INR initial weights, which were further fine-tuned by patient-specific optimization using measurement-domain fidelity and DVF-smoothness losses. We evaluated DiffINR on two tasks: cone-beam computed tomography (CBCT) estimation from limited-angle x-ray projections and real-time volumetric magnetic resonance imaging (MRI) estimation from ultra-sparse k-space data. Main Results. For orthogonal-view 90° CBCT estimation, DiffINR achieved a structural similarity index measure (SSIM) of 0.962 ± 0.021, relative error of 6.84% ± 1.70%, and target registration error of 3.61 ± 1.27 mm, outperforming the iterative methods, population-based models, and randomly initialized INR baselines. For volumetric MRI estimation with 13 radial spokes per slice, DiffINR achieved an SSIM of 0.985 ± 0.007, Dice coefficient of 0.89 ± 0.04, and center-of-mass error of 1.30 ± 1.09 mm. DiffINR showed high robustness to data distribution shifts at test time, including dataset changes, limited-angle setting variations, and different MRI sampling sparsity levels. Test-time runtime was 23 s for CBCT estimation and 27 s for MRI estimation, compared with 795 s and 928 s, respectively, for INR optimization from scratch. Significance. By combining hyperdiffusion-based INR initialization with patient-specific data consistency, DiffINR achieves fast and accurate deformation-driven volumetric image estimation.
- 9 October 2026
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 71  (19) , 195030
Open AccessDiffINR: conditional neural field diffusion for deformation-driven volumetric image estimation from limited measurements
Objective. We propose DiffINR for rapid deformation-driven estimation of patient-specific target volumes from limited measurements. DiffINR uses a coordinate-based implicit neural representation (INR) to estimate a dense deformation vector field (DVF) that warps a high-quality prior volume (source) to the target anatomy, with patient-specific INR initialization provided by a population-based conditional hyperdiffusion model. Approach. During offline training, INRs were optimized on fully sampled source-target image volume pairs to represent DVFs. A Transformer diffusion model, hyperdiffusion, learned the distribution of the resulting INR weights. The model was jointly conditioned on anatomical features extracted from the training source and target volumes by a foundation model and on features encoded directly from limited measurements. At inference, hyperdiffusion generated INR initial weights, which were further fine-tuned by patient-specific optimization using measurement-domain fidelity and DVF-smoothness losses. We evaluated DiffINR on two tasks: cone-beam computed tomography (CBCT) estimation from limited-angle x-ray projections and real-time volumetric magnetic resonance imaging (MRI) estimation from ultra-sparse k-space data. Main Results. For orthogonal-view 90° CBCT estimation, DiffINR achieved a structural similarity index measure (SSIM) of 0.962 ± 0.021, relative error of 6.84% ± 1.70%, and target registration error of 3.61 ± 1.27 mm, outperforming the iterative methods, population-based models, and randomly initialized INR baselines. For volumetric MRI estimation with 13 radial spokes per slice, DiffINR achieved an SSIM of 0.985 ± 0.007, Dice coefficient of 0.89 ± 0.04, and center-of-mass error of 1.30 ± 1.09 mm. DiffINR showed high robustness to data distribution shifts at test time, including dataset changes, limited-angle setting variations, and different MRI sampling sparsity levels. Test-time runtime was 23 s for CBCT estimation and 27 s for MRI estimation, compared with 795 s and 928 s, respectively, for INR optimization from scratch. Significance. By combining hyperdiffusion-based INR initialization with patient-specific data consistency, DiffINR achieves fast and accurate deformation-driven volumetric image estimation.
- Open Access
- 3 September 2026
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 71  (17) , 175018
S2V-DREME: a time-resolved slice-to-volume MR image reconstruction framework with dynamic reconstruction and motion estimation
Objective. Existing volumetric magnetic resonance imaging (MRI) techniques are constrained by the trade-off between acquisition time and image quality, limiting accuracy in motion-impacted sites such as the liver. To enable fast, better-quality volumetric imaging with sufficient spatiotemporal resolution, we developed a time-resolved volumetric MRI technique that recovers 3D volumes from acquired 2D MR slices for real-time 3D anatomy and motion tracking. Approach. 2D MR slices dynamically acquired in time and space were mapped to time-resolved 3D MRIs using a one-shot slice-to-volume framework, S2V-DREME. The model jointly estimates a reference 3D MRI and time-resolved deformation vector fields (DVFs) that warp the reference volume into dynamic 3D MRIs. The reference volume is represented by a spatial implicit neural representation (INR), while the DVFs are derived via low-rank motion modeling. Motion basis components (MBCs) are generated by a spline-enhanced INR (SINR)-based motion generator, with coefficients inferred by a feature-wise linear modulation-based motion encoder. A progressive optimization strategy sequentially initializes the spatial INR and MBCs before joint optimization. The loss function integrates slice data fidelity, total variation regularization, MBC normalization, and DVF smoothness constraints. Main results. S2V-DREME generates time-resolved volumetric MRIs from 2D MR slice inputs. It was evaluated on digital phantom extended cardiac torso (XCAT), physical phantom, and human studies. In XCAT, it accurately captured regular and irregular motion during dynamic reconstruction (training stage, Dice similarity coefficient (DSC)/COME: 0.92 ± 0.03/0.98 ± 0.43 mm) and real-time motion estimation (testing stage, DSC/COME: 0.91 ± 0.02/0.99 ± 0.73 mm). Physical phantom experiments achieved a mean COME of 1.16 mm, and human studies demonstrated the feasibility of time-resolved 3D reconstruction from orthogonal-view and single-view slice acquisitions. Significance. By combining a novel step-and-shoot acquisition protocol with motion-compensated one-shot learning, S2V-DREME enables accurate time-resolved volumetric MRI reconstruction and motion tracking from cineslices, with strong potential for rapid volumetric imaging and real-time MR-guided adaptive radiotherapy.
- 3 September 2026
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 71  (17) , 175018
Open AccessS2V-DREME: a time-resolved slice-to-volume MR image reconstruction framework with dynamic reconstruction and motion estimation
Objective. Existing volumetric magnetic resonance imaging (MRI) techniques are constrained by the trade-off between acquisition time and image quality, limiting accuracy in motion-impacted sites such as the liver. To enable fast, better-quality volumetric imaging with sufficient spatiotemporal resolution, we developed a time-resolved volumetric MRI technique that recovers 3D volumes from acquired 2D MR slices for real-time 3D anatomy and motion tracking. Approach. 2D MR slices dynamically acquired in time and space were mapped to time-resolved 3D MRIs using a one-shot slice-to-volume framework, S2V-DREME. The model jointly estimates a reference 3D MRI and time-resolved deformation vector fields (DVFs) that warp the reference volume into dynamic 3D MRIs. The reference volume is represented by a spatial implicit neural representation (INR), while the DVFs are derived via low-rank motion modeling. Motion basis components (MBCs) are generated by a spline-enhanced INR (SINR)-based motion generator, with coefficients inferred by a feature-wise linear modulation-based motion encoder. A progressive optimization strategy sequentially initializes the spatial INR and MBCs before joint optimization. The loss function integrates slice data fidelity, total variation regularization, MBC normalization, and DVF smoothness constraints. Main results. S2V-DREME generates time-resolved volumetric MRIs from 2D MR slice inputs. It was evaluated on digital phantom extended cardiac torso (XCAT), physical phantom, and human studies. In XCAT, it accurately captured regular and irregular motion during dynamic reconstruction (training stage, Dice similarity coefficient (DSC)/COME: 0.92 ± 0.03/0.98 ± 0.43 mm) and real-time motion estimation (testing stage, DSC/COME: 0.91 ± 0.02/0.99 ± 0.73 mm). Physical phantom experiments achieved a mean COME of 1.16 mm, and human studies demonstrated the feasibility of time-resolved 3D reconstruction from orthogonal-view and single-view slice acquisitions. Significance. By combining a novel step-and-shoot acquisition protocol with motion-compensated one-shot learning, S2V-DREME enables accurate time-resolved volumetric MRI reconstruction and motion tracking from cineslices, with strong potential for rapid volumetric imaging and real-time MR-guided adaptive radiotherapy.
- Open Access
- 19 January 2026
- Published by IOP Publishing in Machine Learning: Health
- Vol. 2  (1) , 015003
TSMS-SAM2: multi-scale temporal sampling augmentation and memory-splitting pruning for promptable video object segmentation and tracking in surgical scenarios
Promptable video object segmentation and tracking (VOST) has seen significant advances with the emergence of foundation models like Segment Anything Model 2 (SAM2); however, their application in surgical video analysis remains challenging due to complex motion dynamics and the redundancy of memory that impedes effective learning. In this work, we propose TSMS-SAM2, a novel framework that enhances promptable VOST in surgical videos by addressing challenges of rapid object motion and memory redundancy in SAM2. TSMS-SAM2 introduces two key strategies: multi-temporal-scale video sampling augmentation to improve robustness against motion variability, and a memory splitting and pruning mechanism that organizes and filters past frame features for more efficient and accurate segmentation. Evaluated on EndoVis2017 and EndoVis2018 datasets, TSMS-SAM2 achieved the highest mean (± s.d.) Dice scores of 95.24 ± 0.96% and 86.73 ± 15.46%, respectively, outperforming prior SAM-based and task-specific methods. Extensive ablation studies confirm the effectiveness of multiscale temporal augmentation and memory splitting, highlighting the framework’s potential for robust, efficient segmentation in complex surgical scenarios. Our source code will be made available at https://github.com/apple1986/TSMS-SAM2.
- 19 January 2026
- Published by IOP Publishing in Machine Learning: Health
- Vol. 2  (1) , 015003
Open AccessTSMS-SAM2: multi-scale temporal sampling augmentation and memory-splitting pruning for promptable video object segmentation and tracking in surgical scenarios
Promptable video object segmentation and tracking (VOST) has seen significant advances with the emergence of foundation models like Segment Anything Model 2 (SAM2); however, their application in surgical video analysis remains challenging due to complex motion dynamics and the redundancy of memory that impedes effective learning. In this work, we propose TSMS-SAM2, a novel framework that enhances promptable VOST in surgical videos by addressing challenges of rapid object motion and memory redundancy in SAM2. TSMS-SAM2 introduces two key strategies: multi-temporal-scale video sampling augmentation to improve robustness against motion variability, and a memory splitting and pruning mechanism that organizes and filters past frame features for more efficient and accurate segmentation. Evaluated on EndoVis2017 and EndoVis2018 datasets, TSMS-SAM2 achieved the highest mean (± s.d.) Dice scores of 95.24 ± 0.96% and 86.73 ± 15.46%, respectively, outperforming prior SAM-based and task-specific methods. Extensive ablation studies confirm the effectiveness of multiscale temporal augmentation and memory splitting, highlighting the framework’s potential for robust, efficient segmentation in complex surgical scenarios. Our source code will be made available at https://github.com/apple1986/TSMS-SAM2.
- Open Access
- 22 December 2025
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 71  (1) , 015003
cMeta-INR: cohort-informed meta-learning-based implicit neural representation for deformable registration-driven real-time volumetric MRI estimation
Objective. Rapid and accurate reconstruction of high-quality three-dimensional magnetic resonance (MR) images from undersampled k -space data with variable sampling patterns remains a challenge due to limited available information and the need to preserve rich anatomical details. Deformable image registration provides a promising solution by warping a fully-sampled reference image to align with undersampled data acquired on-board (from an image-guided treatment delivery platform like MR-LINACs). In this study, we proposed a novel registration framework—cohort-informed meta-learning (cMeta)—to enhance the accuracy and efficiency of implicit neural representations (INR) for limited k -space data-driven patient-specific deformable image registration. Approach. cMeta-INR incorporated token-aware modulation and population-level deformation priors to guide an INR template-based meta-learning process. By encoding contextual information and leveraging cohort-informed priors, the resulting meta-learning framework enabled the INR to rapidly adapt to new registration cases with undersampled k -space data. Specifically, for the meta learning, a modulation module with token-awareness was introduced to modulate the INR template, and a pre-trained population-based registration network (KS-RegNet) was employed to generate coarse, reference deformation vector fields and latent embeddings for computing the deformation discrepancy loss and embedding similarity loss. During test-time adaptation, the INR, initialized from the meta-learned template, was efficiently fine-tuned to new k -space data with minimal iterations. Main results. Experiments were conducted on 14 abdominal and 11 cardiac 4D magnetic resonance imagings (MRIs) with 5–13 spokes. cMeta-INR outperformed state-of-the-art methods, achieving the best average (± s.d.) Dice similarity coefficients (0.778 ± 0.056 for abdominal and 0.894 ± 0.076 for cardiac data), and center-of-mass errors (3.04 ± 1.48 mm and 1.32 ± 1.02 mm, respectively), while enabling rapid test-time adaptation of only ∼35 s on an NVIDIA H100 GPU. Significance. The proposed cohort-informed meta-learning framework effectively enhanced the adaptation capabilities of INRs to individual patients under highly undersampled k -space scenarios, demonstrating strong potential for fast and accurate patient-specific deformable registration.
- 22 December 2025
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 71  (1) , 015003
Open AccesscMeta-INR: cohort-informed meta-learning-based implicit neural representation for deformable registration-driven real-time volumetric MRI estimation
Objective. Rapid and accurate reconstruction of high-quality three-dimensional magnetic resonance (MR) images from undersampled k -space data with variable sampling patterns remains a challenge due to limited available information and the need to preserve rich anatomical details. Deformable image registration provides a promising solution by warping a fully-sampled reference image to align with undersampled data acquired on-board (from an image-guided treatment delivery platform like MR-LINACs). In this study, we proposed a novel registration framework—cohort-informed meta-learning (cMeta)—to enhance the accuracy and efficiency of implicit neural representations (INR) for limited k -space data-driven patient-specific deformable image registration. Approach. cMeta-INR incorporated token-aware modulation and population-level deformation priors to guide an INR template-based meta-learning process. By encoding contextual information and leveraging cohort-informed priors, the resulting meta-learning framework enabled the INR to rapidly adapt to new registration cases with undersampled k -space data. Specifically, for the meta learning, a modulation module with token-awareness was introduced to modulate the INR template, and a pre-trained population-based registration network (KS-RegNet) was employed to generate coarse, reference deformation vector fields and latent embeddings for computing the deformation discrepancy loss and embedding similarity loss. During test-time adaptation, the INR, initialized from the meta-learned template, was efficiently fine-tuned to new k -space data with minimal iterations. Main results. Experiments were conducted on 14 abdominal and 11 cardiac 4D magnetic resonance imagings (MRIs) with 5–13 spokes. cMeta-INR outperformed state-of-the-art methods, achieving the best average (± s.d.) Dice similarity coefficients (0.778 ± 0.056 for abdominal and 0.894 ± 0.076 for cardiac data), and center-of-mass errors (3.04 ± 1.48 mm and 1.32 ± 1.02 mm, respectively), while enabling rapid test-time adaptation of only ∼35 s on an NVIDIA H100 GPU. Significance. The proposed cohort-informed meta-learning framework effectively enhanced the adaptation capabilities of INRs to individual patients under highly undersampled k -space scenarios, demonstrating strong potential for fast and accurate patient-specific deformable registration.
- Open Access
- 8 August 2025
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70  (17)
A dynamic reconstruction and motion estimation framework for cardiorespiratory motion-resolved real-time volumetric MR imaging (DREME-MR)
Objective: Based on a 3D pre-treatment magnetic resonance (MR) scan, we developed DREME-MR to jointly reconstruct the reference patient anatomy and solve a data-driven, patient-specific cardiorespiratory motion model. Via a motion encoder simultaneously learned during the reconstruction, DREME-MR further enables real-time volumetric MR imaging and cardiorespiratory motion tracking with minimal intra-treatment k-space data. Approach: DREME-MR integrates dynamic MRI reconstruction and real-time MR imaging into a unified, dual-task learning framework. From a 3D radial-spoke-based pre-treatment MR scan, DREME-MR uses spatiotemporal implicit-neural-representation (INR) to reconstruct pre-treatment dynamic volumetric MR images (learning task 1). The INR-based reconstruction takes a joint image reconstruction and deformable registration approach, yielding a reference anatomy and a corresponding cardiorespiratory motion model. The motion model adopts a low-rank, multi-resolution representation to approximate motion fields as products of motion coefficients and motion basis components (MBCs). Via a progressive, frequency-guided strategy, DREME-MR decouples cardiac MBCs from respiratory MBCs to resolve the two distinct motion modes. Simultaneously with the pre-treatment dynamic MRI reconstruction, DREME-MR also trains a multilayer perceptron (MLP)-based motion encoder to infer cardiorespiratory motion coefficients directly from the raw k-space data (learning task 2), allowing real-time, intra-treatment volumetric MR imaging and motion tracking with minimal k-space data (20-30 spokes) acquired after the pre-treatment MRI scan. Main results: Evaluated using data from a digital phantom (XCAT) and a human scan, DREME-MR solves real-time 3D cardiorespiratory motion with a latency of < 165 ms (= 150-ms data acquisition + 15-ms inference time), fulfilling the temporal constraint of real-time imaging. The XCAT study achieves mean(±S.D.) center-of-mass tracking errors of 0.73±0.38mm for a lung tumor and 1.69±1.12mm for the left ventricle. The human study shows good motion correlations (liver: 0.96; left ventricle: 0.65) between DREME-MR-solved motion and extracted surrogate signals. Significance: DREME-MR allows real-time 3D MRI and cardiorespiratory motion tracking with low latency, advancing intra-treatment MR-guided adaptive radiotherapy, including real-time multileaf collimator (MLC) tracking.
- 8 August 2025
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70  (17)
Open AccessA dynamic reconstruction and motion estimation framework for cardiorespiratory motion-resolved real-time volumetric MR imaging (DREME-MR)
Objective: Based on a 3D pre-treatment magnetic resonance (MR) scan, we developed DREME-MR to jointly reconstruct the reference patient anatomy and solve a data-driven, patient-specific cardiorespiratory motion model. Via a motion encoder simultaneously learned during the reconstruction, DREME-MR further enables real-time volumetric MR imaging and cardiorespiratory motion tracking with minimal intra-treatment k-space data. Approach: DREME-MR integrates dynamic MRI reconstruction and real-time MR imaging into a unified, dual-task learning framework. From a 3D radial-spoke-based pre-treatment MR scan, DREME-MR uses spatiotemporal implicit-neural-representation (INR) to reconstruct pre-treatment dynamic volumetric MR images (learning task 1). The INR-based reconstruction takes a joint image reconstruction and deformable registration approach, yielding a reference anatomy and a corresponding cardiorespiratory motion model. The motion model adopts a low-rank, multi-resolution representation to approximate motion fields as products of motion coefficients and motion basis components (MBCs). Via a progressive, frequency-guided strategy, DREME-MR decouples cardiac MBCs from respiratory MBCs to resolve the two distinct motion modes. Simultaneously with the pre-treatment dynamic MRI reconstruction, DREME-MR also trains a multilayer perceptron (MLP)-based motion encoder to infer cardiorespiratory motion coefficients directly from the raw k-space data (learning task 2), allowing real-time, intra-treatment volumetric MR imaging and motion tracking with minimal k-space data (20-30 spokes) acquired after the pre-treatment MRI scan. Main results: Evaluated using data from a digital phantom (XCAT) and a human scan, DREME-MR solves real-time 3D cardiorespiratory motion with a latency of < 165 ms (= 150-ms data acquisition + 15-ms inference time), fulfilling the temporal constraint of real-time imaging. The XCAT study achieves mean(±S.D.) center-of-mass tracking errors of 0.73±0.38mm for a lung tumor and 1.69±1.12mm for the left ventricle. The human study shows good motion correlations (liver: 0.96; left ventricle: 0.65) between DREME-MR-solved motion and extracted surrogate signals. Significance: DREME-MR allows real-time 3D MRI and cardiorespiratory motion tracking with low latency, advancing intra-treatment MR-guided adaptive radiotherapy, including real-time multileaf collimator (MLC) tracking.
- Open Access
- 25 July 2025
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70  (16)
Time-resolved dynamic CBCT reconstruction using prior-model-free spatiotemporal Gaussian representation (PMF-STGR)
Objective. Time-resolved CBCT imaging, which reconstructs a dynamic sequence of CBCTs reflecting intra-scan motion (one CBCT per x-ray projection without phase sorting/binning), is highly desired for regular/irregular motion characterization, patient setup, and motion-adapted radiotherapy. Representing patient anatomy and associated motion fields as 3D Gaussians, we developed a Gaussian representation-based framework (PMF-STGR) for fast and accurate dynamic CBCT reconstruction. Approach. PMF-STGR comprises three major components: a dense set of 3D Gaussians to reconstruct a reference-frame CBCT for the dynamic sequence; another 3D Gaussian set to capture three-level, coarse-to-fine motion-basis-components (MBCs) to model the intra-scan motion; and a CNN-based motion encoder to solve projection-specific temporal coefficients for the MBCs. Scaled by the temporal coefficients, the learned MBCs will combine into deformation vector fields (DVFs) to deform the reference CBCT into projection-specific, time-resolved CBCTs to capture the dynamic motion. Due to the strong representation power of 3D Gaussians, PMF-STGR can reconstruct dynamic CBCTs in a ‘one-shot’ training fashion from a standard 3D CBCT scan, without using any prior anatomical/motion model. Main results. We evaluated PMF-STGR using XCAT phantom simulations and real patient full/half-fan scans. Metrics, including the image relative error (RE), structural-similarity-index-measure (SSIM), tumor center-of-mass-error (COME), and landmark localization error (LE), were used to evaluate the accuracy of solved dynamic CBCTs and motion. PMF-STGR shows clear advantages over a state-of-the-art, implicit neural representation (INR)-based approach, PMF-STINR. Compared with PMF-STINR, PMF-STGR reduces reconstruction time by ~50% while reconstructing less blurred images with comparable/better motion accuracy. For XCAT, the mean(±s.d.) RE, SSIM, and COME were 0.128(0.009), 0.990(0.002), and 0.71mm(0.40mm) for PMF-STGR, compared with 0.149(0.016), 0.944(0.006), and 0.94mm(0.18mm) for PMF-STINR. For patients, the mean(±s.d.) landmark LE was 1.40mm(0.34mm) for PMF-STGR, and 1.54mm(0.35mm) for PMF-STINR. Significance. With improved efficiency/accuracy, PMF-STGR enhances the applicability of dynamic CBCT imaging for potential clinical translation.
- 25 July 2025
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70  (16)
Open AccessTime-resolved dynamic CBCT reconstruction using prior-model-free spatiotemporal Gaussian representation (PMF-STGR)
Objective. Time-resolved CBCT imaging, which reconstructs a dynamic sequence of CBCTs reflecting intra-scan motion (one CBCT per x-ray projection without phase sorting/binning), is highly desired for regular/irregular motion characterization, patient setup, and motion-adapted radiotherapy. Representing patient anatomy and associated motion fields as 3D Gaussians, we developed a Gaussian representation-based framework (PMF-STGR) for fast and accurate dynamic CBCT reconstruction. Approach. PMF-STGR comprises three major components: a dense set of 3D Gaussians to reconstruct a reference-frame CBCT for the dynamic sequence; another 3D Gaussian set to capture three-level, coarse-to-fine motion-basis-components (MBCs) to model the intra-scan motion; and a CNN-based motion encoder to solve projection-specific temporal coefficients for the MBCs. Scaled by the temporal coefficients, the learned MBCs will combine into deformation vector fields (DVFs) to deform the reference CBCT into projection-specific, time-resolved CBCTs to capture the dynamic motion. Due to the strong representation power of 3D Gaussians, PMF-STGR can reconstruct dynamic CBCTs in a ‘one-shot’ training fashion from a standard 3D CBCT scan, without using any prior anatomical/motion model. Main results. We evaluated PMF-STGR using XCAT phantom simulations and real patient full/half-fan scans. Metrics, including the image relative error (RE), structural-similarity-index-measure (SSIM), tumor center-of-mass-error (COME), and landmark localization error (LE), were used to evaluate the accuracy of solved dynamic CBCTs and motion. PMF-STGR shows clear advantages over a state-of-the-art, implicit neural representation (INR)-based approach, PMF-STINR. Compared with PMF-STINR, PMF-STGR reduces reconstruction time by ~50% while reconstructing less blurred images with comparable/better motion accuracy. For XCAT, the mean(±s.d.) RE, SSIM, and COME were 0.128(0.009), 0.990(0.002), and 0.71mm(0.40mm) for PMF-STGR, compared with 0.149(0.016), 0.944(0.006), and 0.94mm(0.18mm) for PMF-STINR. For patients, the mean(±s.d.) landmark LE was 1.40mm(0.34mm) for PMF-STGR, and 1.54mm(0.35mm) for PMF-STINR. Significance. With improved efficiency/accuracy, PMF-STGR enhances the applicability of dynamic CBCT imaging for potential clinical translation.
- Open Access
- 22 June 2025
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70  (12) , 125021
A conditional point cloud diffusion model for deformable liver motion tracking via a single arbitrarily-angled x-ray projection
Objective. Deformable liver motion tracking using a single x-ray projection enables real-time motion monitoring and treatment intervention. We introduce a conditional point cloud diffusion (PCD) model-based framework for accurate and robust liver motion tracking from arbitrarily angled single x-ray projections. Approach. We propose a conditional PCD model for liver motion tracking (PCD-Liver), which estimates volumetric liver motion by solving deformable vector fields (DVFs) of a prior liver surface point cloud, based on a single x-ray image. It is a patient-specific model of two main components: a rigid alignment model to estimate the liver’s overall shifts, and a conditional PCD model that further corrects for the liver surface’s deformation. Conditioned on the motion-encoded features extracted from a single x-ray projection by a geometry-informed feature pooling layer, the diffusion model iteratively solves detailed liver surface DVFs in a projection angle-agnostic fashion. The liver surface motion solved by PCD-Liver is subsequently fed as the boundary condition into a U-Net-based biomechanical model to infer the liver’s internal motion to localize liver tumors. A dataset of 10 liver cancer patients was used for evaluation. We used the root mean square error (RMSE) and 95-percentile Hausdorff distance (HD95) metrics to examine the liver point cloud motion estimation accuracy, and the center-of-mass error (COME) to quantify the liver tumor localization error. Main Results. The mean (±s.d.) RMSE, HD95, and COME of the prior liver or tumor before motion estimation were 8.82 mm (±3.58 mm), 10.84 mm (±4.55 mm), and 9.72 mm (±4.34 mm), respectively. After PCD-Liver’s motion estimation, the corresponding values were 3.63 mm (±1.88 mm), 4.29 mm (±1.75 mm), and 3.46 mm (±2.15 mm). Under highly noisy conditions, PCD-Liver maintained stable performance. Significance. This study presents an accurate and robust framework for liver deformable motion estimation and tumor localization for image-guided radiotherapy.
- 22 June 2025
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70  (12) , 125021
Open AccessA conditional point cloud diffusion model for deformable liver motion tracking via a single arbitrarily-angled x-ray projection
Objective. Deformable liver motion tracking using a single x-ray projection enables real-time motion monitoring and treatment intervention. We introduce a conditional point cloud diffusion (PCD) model-based framework for accurate and robust liver motion tracking from arbitrarily angled single x-ray projections. Approach. We propose a conditional PCD model for liver motion tracking (PCD-Liver), which estimates volumetric liver motion by solving deformable vector fields (DVFs) of a prior liver surface point cloud, based on a single x-ray image. It is a patient-specific model of two main components: a rigid alignment model to estimate the liver’s overall shifts, and a conditional PCD model that further corrects for the liver surface’s deformation. Conditioned on the motion-encoded features extracted from a single x-ray projection by a geometry-informed feature pooling layer, the diffusion model iteratively solves detailed liver surface DVFs in a projection angle-agnostic fashion. The liver surface motion solved by PCD-Liver is subsequently fed as the boundary condition into a U-Net-based biomechanical model to infer the liver’s internal motion to localize liver tumors. A dataset of 10 liver cancer patients was used for evaluation. We used the root mean square error (RMSE) and 95-percentile Hausdorff distance (HD95) metrics to examine the liver point cloud motion estimation accuracy, and the center-of-mass error (COME) to quantify the liver tumor localization error. Main Results. The mean (±s.d.) RMSE, HD95, and COME of the prior liver or tumor before motion estimation were 8.82 mm (±3.58 mm), 10.84 mm (±4.55 mm), and 9.72 mm (±4.34 mm), respectively. After PCD-Liver’s motion estimation, the corresponding values were 3.63 mm (±1.88 mm), 4.29 mm (±1.75 mm), and 3.46 mm (±2.15 mm). Under highly noisy conditions, PCD-Liver maintained stable performance. Significance. This study presents an accurate and robust framework for liver deformable motion estimation and tumor localization for image-guided radiotherapy.
- Open Access
- 7 April 2025
- Published by IOP Publishing in Machine Learning: Science and Technology
- Vol. 6  (2) , 025007
FDDM: unsupervised medical image translation with a frequency-decoupled diffusion model
Diffusion models have demonstrated significant potential in producing high-quality images in medical image translation to aid disease diagnosis, localization, and treatment. Nevertheless, current diffusion models often fall short when it comes to faithfully translating medical images. They struggle to accurately preserve anatomical structures, especially when working with unpaired datasets. In this study, we introduce the frequency decoupled diffusion model (FDDM) for magnetic resonance (MR)-to-computed tomography (CT) conversion. The differences between MR and CT images lie in both anatomical structures (e.g. the outlines of organs or bones) and the data distribution (e.g. intensity values and contrast within). Therefore, FDDM first converts anatomical information using an initial conversion module. Then, the converted anatomical information guides a subsequent diffusion model to generate high-quality CT images. Our diffusion model uses a dual-path reverse diffusion process for low-frequency and high-frequency information, achieving a better balance between image quality and anatomical accuracy. We extensively evaluated FDDM using two public datasets for brain MR-to-CT and pelvis MR-to-CT translations. The results show that FDDM outperforms generative adversarial network (GAN)-based, variational autoencoder (VAE)-based, and diffusion-based models. The evaluation metrics included Fréchet inception distance (FID), mean absolute error, mean squared error, structural similarity index measure, and Dice similarity coefficient (DICE). FDDM achieved the best scores on all metrics for both datasets, particularly excelling in FID, with scores of 25.9 for brain data and 29.2 for pelvis data, significantly outperforming the other methods. These results demonstrate that FDDM can generate high-quality target domain images while maintaining the accuracy of translated anatomical structures, thereby facilitating more precise/accurate downstream tasks including anatomy segmentation and radiotherapy planning.
- 7 April 2025
- Published by IOP Publishing in Machine Learning: Science and Technology
- Vol. 6  (2) , 025007
Open AccessFDDM: unsupervised medical image translation with a frequency-decoupled diffusion model
Diffusion models have demonstrated significant potential in producing high-quality images in medical image translation to aid disease diagnosis, localization, and treatment. Nevertheless, current diffusion models often fall short when it comes to faithfully translating medical images. They struggle to accurately preserve anatomical structures, especially when working with unpaired datasets. In this study, we introduce the frequency decoupled diffusion model (FDDM) for magnetic resonance (MR)-to-computed tomography (CT) conversion. The differences between MR and CT images lie in both anatomical structures (e.g. the outlines of organs or bones) and the data distribution (e.g. intensity values and contrast within). Therefore, FDDM first converts anatomical information using an initial conversion module. Then, the converted anatomical information guides a subsequent diffusion model to generate high-quality CT images. Our diffusion model uses a dual-path reverse diffusion process for low-frequency and high-frequency information, achieving a better balance between image quality and anatomical accuracy. We extensively evaluated FDDM using two public datasets for brain MR-to-CT and pelvis MR-to-CT translations. The results show that FDDM outperforms generative adversarial network (GAN)-based, variational autoencoder (VAE)-based, and diffusion-based models. The evaluation metrics included Fréchet inception distance (FID), mean absolute error, mean squared error, structural similarity index measure, and Dice similarity coefficient (DICE). FDDM achieved the best scores on all metrics for both datasets, particularly excelling in FID, with scores of 25.9 for brain data and 29.2 for pelvis data, significantly outperforming the other methods. These results demonstrate that FDDM can generate high-quality target domain images while maintaining the accuracy of translated anatomical structures, thereby facilitating more precise/accurate downstream tasks including anatomy segmentation and radiotherapy planning.
- Open Access
- 19 January 2025
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70  (2) , 025026
Real-time CBCT imaging and motion tracking via a single arbitrarily-angled x-ray projection by a joint dynamic reconstruction and motion estimation (DREME) framework
Objective. Real-time cone-beam computed tomography (CBCT) provides instantaneous visualization of patient anatomy for image guidance, motion tracking, and online treatment adaptation in radiotherapy. While many real-time imaging and motion tracking methods leveraged patient-specific prior information to alleviate under-sampling challenges and meet the temporal constraint (Approach. DREME incorporates a deep learning-based real-time CBCT imaging and motion estimation method into a dynamic CBCT reconstruction framework. The reconstruction framework reconstructs a dynamic sequence of CBCTs in a data-driven manner from a standard pre-treatment scan, without requiring patient-specific prior knowledge. Meanwhile, a convolutional neural network-based motion encoder is jointly trained during the reconstruction to learn motion-related features relevant for real-time motion estimation, based on a single arbitrarily-angled x-ray projection. DREME was tested on digital phantom simulations and real patient studies. Main Results. DREME accurately solved 3D respiration-induced anatomical motion in real time (∼1.5 ms inference time for each x-ray projection). For the digital phantom studies, it achieved an average lung tumor center-of-mass localization error of 1.2 ± 0.9 mm (Mean ± SD). For the patient studies, it achieved a real-time tumor localization accuracy of 1.6 ± 1.6 mm in the projection domain. Significance. DREME achieves CBCT and volumetric motion estimation in real time from a single x-ray projection at arbitrary angles, paving the way for future clinical applications in intra-fractional motion management. In addition, it can be used for dose tracking and treatment assessment, when combined with real-time dose calculation.
- 19 January 2025
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 70  (2) , 025026
Open AccessReal-time CBCT imaging and motion tracking via a single arbitrarily-angled x-ray projection by a joint dynamic reconstruction and motion estimation (DREME) framework
Objective. Real-time cone-beam computed tomography (CBCT) provides instantaneous visualization of patient anatomy for image guidance, motion tracking, and online treatment adaptation in radiotherapy. While many real-time imaging and motion tracking methods leveraged patient-specific prior information to alleviate under-sampling challenges and meet the temporal constraint (Approach. DREME incorporates a deep learning-based real-time CBCT imaging and motion estimation method into a dynamic CBCT reconstruction framework. The reconstruction framework reconstructs a dynamic sequence of CBCTs in a data-driven manner from a standard pre-treatment scan, without requiring patient-specific prior knowledge. Meanwhile, a convolutional neural network-based motion encoder is jointly trained during the reconstruction to learn motion-related features relevant for real-time motion estimation, based on a single arbitrarily-angled x-ray projection. DREME was tested on digital phantom simulations and real patient studies. Main Results. DREME accurately solved 3D respiration-induced anatomical motion in real time (∼1.5 ms inference time for each x-ray projection). For the digital phantom studies, it achieved an average lung tumor center-of-mass localization error of 1.2 ± 0.9 mm (Mean ± SD). For the patient studies, it achieved a real-time tumor localization accuracy of 1.6 ± 1.6 mm in the projection domain. Significance. DREME achieves CBCT and volumetric motion estimation in real time from a single x-ray projection at arbitrary angles, paving the way for future clinical applications in intra-fractional motion management. In addition, it can be used for dose tracking and treatment assessment, when combined with real-time dose calculation.
- Open Access
- 26 June 2024
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 69  (13) , 135008
Prior frequency guided diffusion model for limited angle (LA)-CBCT reconstruction
Objective. Cone-beam computed tomography (CBCT) is widely used in image-guided radiotherapy. Reconstructing CBCTs from limited-angle acquisitions (LA-CBCT) is highly desired for improved imaging efficiency, dose reduction, and better mechanical clearance. LA-CBCT reconstruction, however, suffers from severe under-sampling artifacts, making it a highly ill-posed inverse problem. Diffusion models can generate data/images by reversing a data-noising process through learned data distributions; and can be incorporated as a denoiser/regularizer in LA-CBCT reconstruction. In this study, we developed a diffusion model-based framework, prior frequency-guided diffusion model (PFGDM), for robust and structure-preserving LA-CBCT reconstruction. Approach. PFGDM uses a conditioned diffusion model as a regularizer for LA-CBCT reconstruction, and the condition is based on high-frequency information extracted from patient-specific prior CT scans which provides a strong anatomical prior for LA-CBCT reconstruction. Specifically, we developed two variants of PFGDM (PFGDM-A and PFGDM-B) with different conditioning schemes. PFGDM-A applies the high-frequency CT information condition until a pre-optimized iteration step, and drops it afterwards to enable both similar and differing CT/CBCT anatomies to be reconstructed. PFGDM-B, on the other hand, continuously applies the prior CT information condition in every reconstruction step, while with a decaying mechanism, to gradually phase out the reconstruction guidance from the prior CT scans. The two variants of PFGDM were tested and compared with current available LA-CBCT reconstruction solutions, via metrics including peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM). Main results. PFGDM outperformed all traditional and diffusion model-based methods. The mean(s.d.) PSNR/SSIM were 27.97(3.10)/0.949(0.027), 26.63(2.79)/0.937(0.029), and 23.81(2.25)/0.896(0.036) for PFGDM-A, and 28.20(1.28)/0.954(0.011), 26.68(1.04)/0.941(0.014), and 23.72(1.19)/0.894(0.034) for PFGDM-B, based on 120°, 90°, and 30° orthogonal-view scan angles respectively. In contrast, the PSNR/SSIM was 19.61(2.47)/0.807(0.048) for 30° for DiffusionMBIR, a diffusion-based method without prior CT conditioning. Significance. PFGDM reconstructs high-quality LA-CBCTs under very-limited gantry angles, allowing faster and more flexible CBCT scans with dose reductions.
- 26 June 2024
- Published by IOP Publishing in Physics in Medicine & Biology
- Vol. 69  (13) , 135008
Open AccessPrior frequency guided diffusion model for limited angle (LA)-CBCT reconstruction
Objective. Cone-beam computed tomography (CBCT) is widely used in image-guided radiotherapy. Reconstructing CBCTs from limited-angle acquisitions (LA-CBCT) is highly desired for improved imaging efficiency, dose reduction, and better mechanical clearance. LA-CBCT reconstruction, however, suffers from severe under-sampling artifacts, making it a highly ill-posed inverse problem. Diffusion models can generate data/images by reversing a data-noising process through learned data distributions; and can be incorporated as a denoiser/regularizer in LA-CBCT reconstruction. In this study, we developed a diffusion model-based framework, prior frequency-guided diffusion model (PFGDM), for robust and structure-preserving LA-CBCT reconstruction. Approach. PFGDM uses a conditioned diffusion model as a regularizer for LA-CBCT reconstruction, and the condition is based on high-frequency information extracted from patient-specific prior CT scans which provides a strong anatomical prior for LA-CBCT reconstruction. Specifically, we developed two variants of PFGDM (PFGDM-A and PFGDM-B) with different conditioning schemes. PFGDM-A applies the high-frequency CT information condition until a pre-optimized iteration step, and drops it afterwards to enable both similar and differing CT/CBCT anatomies to be reconstructed. PFGDM-B, on the other hand, continuously applies the prior CT information condition in every reconstruction step, while with a decaying mechanism, to gradually phase out the reconstruction guidance from the prior CT scans. The two variants of PFGDM were tested and compared with current available LA-CBCT reconstruction solutions, via metrics including peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM). Main results. PFGDM outperformed all traditional and diffusion model-based methods. The mean(s.d.) PSNR/SSIM were 27.97(3.10)/0.949(0.027), 26.63(2.79)/0.937(0.029), and 23.81(2.25)/0.896(0.036) for PFGDM-A, and 28.20(1.28)/0.954(0.011), 26.68(1.04)/0.941(0.014), and 23.72(1.19)/0.894(0.034) for PFGDM-B, based on 120°, 90°, and 30° orthogonal-view scan angles respectively. In contrast, the PSNR/SSIM was 19.61(2.47)/0.807(0.048) for 30° for DiffusionMBIR, a diffusion-based method without prior CT conditioning. Significance. PFGDM reconstructs high-quality LA-CBCTs under very-limited gantry angles, allowing faster and more flexible CBCT scans with dose reductions.