Di Xu

Objectives: Computed tomography (CT) provides high spatial-resolution visualization of 3D structures for various applications. Traditional analytical/iterative CT reconstruction algorithms require hundreds of angular samplings, a condition may not be met practically for physical and mechanical limitations. Sparse view CT reconstruction has been proposed using constrained optimization and machine learning methods with varying success, less so for ultra-sparse view reconstruction. Neural radiance field (NeRF) is a powerful tool for reconstructing and rendering 3D natural scenes from sparse views, but its direct application to 3D medical image reconstruction has been minimally successful due to the differences in photon transportation and available prior information between optic and X-ray. Methods: We develop TomoGRAF to reconstruct high-quality 3D CT volumes using ultra-sparse projections. TomoGRAF has two main novelties pertinent to X-ray physics and CT imaging. First, TomoGRAF’s volume rendering module accumulates x-ray material attenuation passing through an object with CT geometry rather than visible light material color and opacity from surface interaction in NeRF. Second, TomoGRAF penalizes the difference between the simulated and ground truth volume during training besides the 2D views, thus significantly improving the prior fidelity. Results: TomoGRAF is trained on LIDC-IDRI dataset (1011 scans) and evaluated on an unseen in-house dataset (100 scans) of distinct imaging characteristics from training and demonstrates a vast leap in performance compared with state-of-the-art deep learning and NeRF methods. Conclusion: TomoGRAF provides the first generalizable solution for image-guided radiotherapy and interventional radiology applications, where only one/a few X-ray views are available, but 3D volumetric information is desired.

Purpose: 4D MRI with high spatiotemporal resolution is vital to characterize the tumor/tumor motion for liver radiotherapy. However, high-quality 4D MRI requires an impractically long scanning time for dense k-space signal acquisition covering all respiratory phases. Accelerated sparse sampling followed by reconstruction enhancement is desired but often results in degraded image quality and long reconstruction time. We hereby propose the chained iterative reconstruction network (CIRNet) for efficient sparse-sampling reconstruction while maintaining clinically deployable quality. Methods: CIRNet adopts the denoising diffusion probabilistic framework to condition the image reconstruction through a stochastic iterative denoising process. During training, a forward Markovian diffusion process is designed to gradually add Gaussian noise to the densely sampled ground truth (GT), while CIRNet is optimized to iteratively reverse the Markovian process from the forward outputs. At the inference stage, CIRNet performs the reverse process solely to recover signals from noise, conditioned upon the undersampled input. CIRNet is structured with a U-Net architecture, optimized to minimize the L₂ difference between estimated and GT noises. CIRNet processed the 4D data (3D+t) as temporal slices (2D+t). The proposed framework is evaluated on a data cohort consisting of 48 patients (12332 temporal slices) who underwent free-breathing liver 4D MRI. 3-, 6-, 10-, 20- and 30-times acceleration were examined with a retrospective random undersampling scheme. Compressed sensing (CS) reconstruction with a spatiotemporal constraint and a recently proposed deep network, Re-Con-GAN, are selected as baselines. Results: CIRNet consistently achieved superior performance compared to CS and Re-Con-GAN (e.g., PNSR of CIRNet, CS and Re-Con-GAN is at 22.35±2.94, 13.27±3.89 and 13.27±3.89dB in 30 times acceleration). The inference time of CIRNet, CS, and Re-Con-GAN are 11s, 120s, and 0.15s. Conclusion: A novel framework, CIRNet, operating under stochastic iterative refinement for accelerated MR reconstruction, is presented. Compared with published methods, CIRNet maintains useable image quality for acceleration up to 30 times, significantly reducing the burden of 4DMRI.

Objective. Lung tumors can be obscured in x-rays, preventing accurate and robust localization. To improve lung conspicuity for image-guided procedures, we isolate the lungs in the anterior-posterior (AP) x-rays using a lung extraction network (LeX-net) that virtually removes overlapping thoracic structures, including ribs, diaphragm, liver, heart, and trachea. Approach. 73 965 thoracic 3DCTs and 106 thoracic 4DCTs were included. The 3D lung volume was extracted using an open-source lung volume segmentation model. AP digitally reconstructed radiographs (DRRs) of the full anatomy CT and extracted lungs were computed as the input and reference to train a network (LeX-net) to generate lung-extracted DRRs (LeX-net DRRs) from full anatomy DRRs, which adopted a Swin-UNet model with conditional GAN. Subsequently, the trained LeX-net on 3DCT was applied to 4DCT-derived DRRs. Lung tumor tracking was then performed on DRRs using a template-matching method on a holdoff 4DCT test set of 79 patients whose gross tumor volumes were smaller than 20 cm³. Main results. LeX-net successfully isolated the lungs in DRRs, achieving an SSIM of 0.9581 ± 0.0151 and a PSNR of 30.78 ± 2.50 on the testing set of 3DCT-derived DRRs. Its performance declined slightly when applied to the 4DCT but maintained useable lung-only 2D views. On the challenging test set including cases of organ overlap, high tumor mobility, and small tumor size, the individual tumor tracking error for LeX-net DRRs was 0.97 ± 0.86 mm, significantly lower than that of 3.13 ± 5.82 mm using the full anatomy DRRs. LeX-net improved success rates of using 5 mm, 3 mm, and 1 mm tracking windows from 88.1%, 80.0%, and 58.7% to 98.1%, 94.2%, and 73.8%, respectively. Significance. LeX-net removes overlapping anatomies and enhances visualization of the lungs in x-rays. The model trained using 3DCTs is generalizable to 4DCT-derived DRRs, achieving significantly improved tumor tracking outcome.

Image-guided mouse irradiation is essential to understand interventions involving radiation prior to human studies. Our objective is to employ Swin UNEt TRansformers (Swin UNETR) to segment native micro-CT and contrast-enhanced micro-CT scans and benchmark the results against 3D no-new-Net (nnU-Net). Swin UNETR reformulates mouse organ segmentation as a sequence-to-sequence prediction task using a hierarchical Swin Transformer encoder to extract features at five resolution levels, and it connects to a Fully Convolutional Neural Network (FCNN)-based decoder via skip connections. The models were trained and evaluated on open datasets, with data separation based on individual mice. Further evaluation on an external mouse dataset acquired on a different micro-CT with lower kVp and higher imaging noise was also employed to assess model robustness and generalizability. The results indicate that Swin UNETR consistently outperforms nnU-Net and AIMOS in terms of the average dice similarity coefficient (DSC) and the Hausdorff distance (HD95p), except in two mice for intestine contouring. This superior performance is especially evident in the external dataset, confirming the model’s robustness to variations in imaging conditions, including noise and quality, and thereby positioning Swin UNETR as a highly generalizable and efficient tool for automated contouring in pre-clinical workflows.

Objective. Auto-segmentation in mouse micro-CT enhances the efficiency and consistency of preclinical experiments but often struggles with low-native-contrast and morphologically complex organs, such as the spleen, resulting in poor segmentation performance. While CT contrast agents can improve organ conspicuity, their use complicates experimental protocols and reduces feasibility. We developed a 3D Cycle Generative Adversarial Network (CycleGAN) incorporating anatomy-constrained U-Net models to leverage contrast-enhanced CT (CECT) insights to improve unenhanced native CT (NACT) segmentation. Approach. We employed a standard CycleGAN with an anatomical loss function to synthesize virtual CECT images from unpaired NACT scans at two different resolutions. Prior to training, two U-Nets were trained to automatically segment six major organs in NACT and CECT datasets, respectively. These pretrained 3D U-Nets were integrated during the CycleGAN training, segmenting synthetic images, and comparing them against ground truth annotations. The compound loss within the CycleGAN maintained anatomical fidelity. Full image processing was achieved for low-resolution datasets, while high-resolution datasets employed a patch-based method due to GPU memory constraints. Automated segmentation was applied to original NACT and synthetic CECT scans to evaluate CycleGAN performance using the Dice Similarity Coefficient (DSC) and the 95th percentile Hausdorff Distance (HD_95p). Main results. High-resolution scans showed improved auto-segmentation, with an average DSC increase from 0.728 to 0.773 and a reduced HD95p from 1.19 mm to 0.94 mm. Low-resolution scans benefited more from synthetic contrast, showing a DSC increase from 0.586 to 0.682 and an HD_95p reduction from 3.46 mm to 1.24 mm. Significance. Implementing CycleGAN to synthesize CECT scans substantially improved the visibility of the mouse spleen, leading to more precise auto-segmentation. This approach shows the potential in preclinical imaging studies where contrast agent use is impractical.

Purpose. 4D MRI with high spatiotemporal resolution is desired for image-guided liver radiotherapy. Acquiring densely sampling k-space data is time-consuming. Accelerated acquisition with sparse samples is desirable but often causes degraded image quality or long reconstruction time. We propose the Reconstruct Paired Conditional Generative Adversarial Network (Re-Con-GAN) to shorten the 4D MRI reconstruction time while maintaining the reconstruction quality. Methods. Patients who underwent free-breathing liver 4D MRI were included in the study. Fully- and retrospectively under-sampled data at 3, 6 and 10 times (3×, 6× and 10×) were first reconstructed using the nuFFT algorithm. Re-Con-GAN then trained input and output in pairs. Three types of networks, ResNet9, UNet and reconstruction swin transformer (RST), were explored as generators. PatchGAN was selected as the discriminator. Re-Con-GAN processed the data (3D + t) as temporal slices (2D + t). A total of 48 patients with 12 332 temporal slices were split into training (37 patients with 10 721 slices) and test (11 patients with 1611 slices). Compressed sensing (CS) reconstruction with spatiotemporal sparsity constraint was used as a benchmark. Reconstructed image quality was further evaluated with a liver gross tumor volume (GTV) localization task using Mask-RCNN trained from a separate 3D static liver MRI dataset (70 patients; 103 GTV contours). Results. Re-Con-GAN consistently achieved comparable/better PSNR, SSIM, and RMSE scores compared to CS/UNet models. The inference time of Re-Con-GAN, UNet and CS are 0.15, 0.16, and 120 s. The GTV detection task showed that Re-Con-GAN and CS, compared to UNet, better improved the dice score (3× Re-Con-GAN 80.98%; 3× CS 80.74%; 3× UNet 79.88%) of unprocessed under-sampled images (3× 69.61%). Conclusion. A generative network with adversarial training is proposed with promising and efficient reconstruction results demonstrated on an in-house dataset. The rapid and qualitative reconstruction of 4D liver MR has the potential to facilitate online adaptive MR-guided radiotherapy for liver cancer.

Pulmonary nodules may be an early manifestation of lung cancer, the leading cause of cancer-related deaths among both men and women. Numerous studies have established that deep learning methods can yield high-performance levels in the detection of lung nodules in chest X-rays. However, the lack of gold-standard public datasets slows down the progression of the research and prevents benchmarking of methods for this task. To address this, we organized a public research challenge, NODE21, aimed at the detection and generation of lung nodules in chest X-rays. While the detection track assesses state-of-the-art nodule detection systems, the generation track determines the utility of nodule generation algorithms to augment training data and hence improve the performance of the detection systems. This paper summarizes the results of the NODE21 challenge and performs extensive additional experiments to examine the impact of the synthetically generated nodule training images on the detection algorithm performance.

Dual-energy computed tomography (DECT) utilizes separate X-ray energy spectra to improve multi-material decomposition (MMD) for various diagnostic applications. However accurate decomposing more than two types of material remains challenging using conventional methods. Deep learning (DL) methods have shown promise to improve the MMD performance, but typical approaches of conducing DL-MMD in the image domain fail to fully utilize projection information or under iterative setup are computationally inefficient in both training and prediction. In this work, we present a clinical-applicable MMD (>2) framework rFast-MMDNet, operating with raw projection data in non-recursive setup, for breast tissue differentiation. rFast-MMDNet is a two-stage algorithm, including stage-one SinoNet to perform dual energy projection decomposition on tissue sinograms and stage-two FBP-DenoiseNet to perform domain adaptation and image post-processing. rFast-MMDNet was tested on a 2022 DL-Spectral-Challenge breast phantom dataset. The two stages of rFast-MMDNet were evaluated separately and then compared with four noniterative reference methods including a direct inversion method (AA-MMD), an image domain DL method (ID-UNet), AA-MMD/ID-UNet + DenoiseNet and a sinogram domain DL method (Triple-CBCT). Our results show that models trained from information stored in DE transmission domain can yield high-fidelity decomposition of the adipose, calcification, and fibroglandular materials with averaged RMSE, MAE, negative PSNR, and SSIM of 0.004+/-~0, 0.001+/-~0, -45.027+/-~0.542, and 0.002+/-~0 benchmarking to the ground truth, respectively. Training of entire rFast-MMDNet on a 4xRTX A6000 GPU cluster took a day with inference time <1s. All DL methods generally led to more accurate MMD than AA-MMD. rFast-MMDNet outperformed Triple-CBCT, but both are superior to the image-domain based methods.