“See” through the surface: Surface-derived Three-dimensional AI-driven Real-time Imaging Solution for Intra-Treatment Image Guidance
Open Access
- 5 January 2026
- journal article
- Published by IOP Publishing in Machine Learning: Health
- Vol. 2 (1)
Abstract
Respiratory motion is a long-standing challenge for lung stereotactic body radiotherapy (SBRT), particularly for centrally located lung tumors where increased toxicity demands more precise motion management during treatment. Current two-dimensional imaging approaches are insufficient for 3D tumor deformable motion tracking. In this study, we developed and evaluated a Surface-derived Three-dimensional (3D) AI-driven Real-time (STAR) imaging system by transforming a surface imaging system into a 3D real-time imaging solution. STAR integrated two key components: (1) prior-model-free spatiotemporal implicit neural representation (PMF-STINR): a machine-learning sub-system for pretreatment dynamic cone-beam CT (CBCT) reconstruction and motion modeling; and (2) surface-to-deformation network (Surf2DefNet): a deep-learning model that correlates intra-treatment body surface images with internal 3D anatomy and motion fields, trained based on the dynamic CBCT and motion model output of PMF-STINR. Specifically, PMF-STINR reconstructs a reference CBCT and solves an eigenvector-based motion model from a pretreatment CBCT, while Surf2DefNet predicts the motion eigen-vector weightings from surface images, enabling it to infer real-time CBCTs and motion vector fields (MVFs) using intra-treatment surface maps acquired later. We evaluated STAR imaging using both a digital extended cardiac torso (XCAT) phantom with regular and irregular motion patterns and ten patient datasets. The relative error (RE), center-of-mass error (COME), 95th percentile Hausdorff distance (HD95), Dice coefficients (DICE), and Pearson correlation coefficient (PCC) metrics between STAR images and the ‘GT’ were evaluated. For the XCAT phantom and the ten patients, the mean COME values are within 1 mm for all but one patient (1.3 mm). The RE values were consistently low, and the DICE and PCC values exceeded 0.89 in all cases. The HD95 are all within 2 mm except for one patient (2. 78 mm). These results demonstrate that the STAR imaging system can achieve accurate spatiotemporal reconstructions from surface images, providing CBCTs and MVFs for intra-treatment real-time image-guidance, and has the potential to improve safety and efficacy of SBRT for lung cancer.Keywords
Funding Information
- National Institutes of Health (R01 CA240808)
- National Institutes of Health (R01 CA258987)
- National Institutes of Health (R01 CA280135)
- National Institutes of Health (R01 EB034691)
This publication has 48 references indexed in Scilit:
- Deep learning‐based estimation of respiration‐induced deformation from surface motion: A proof‐of‐concept study on 4D thoracic image synthesisMedical Physics, 2025
- DE-NAF: decoupled neural attenuation fields for sparse-view CBCT reconstructionPattern Analysis and Applications, 2025
- Real-time estimation of lung deformation from body surface using a general CoordConv CNNComputer Methods and Programs in Biomedicine, 2023
- Deep learning‐based fast volumetric imaging using kV and MV projection images for lung cancer radiotherapy: A feasibility studyMedical Physics, 2023
- Deep learning‐based synthetization of real‐time in‐treatment 4D images using surface motion and pretreatment images: A proof‐of‐concept studyMedical Physics, 2022
- AAPM Task Group 264: The safe clinical implementation of MLC tracking in radiotherapyMedical Physics, 2020
- SPARE: Sparse‐view reconstruction challenge for 4D cone‐beam CT from a 1‐min scanMedical Physics, 2019
- Reducing the tracking drift of an uncontoured tumor for a portal-image-based dynamically adapted conformal radiotherapy treatmentMedical & Biological Engineering & Computing, 2019
- Feasibility of predicting tumor motion using online data acquired during treatment and a generalized neural network optimized with offline patient tumor trajectoriesMedical Physics, 2018
- Motion management strategies and technical issues associated with stereotactic body radiotherapy of thoracic and upper abdominal tumors: A review from NRG oncologyMedical Physics, 2017