Chuang Niu

AI foundation models (FMs) are transforming cancer management and research. Here, we analyze the state-of-the-art FMs, discuss their impact on cancer management, and argue that the next generation of cancer AI FMs will be defined by multimodality, enhanced reasoning, maximized openness, and sustained human guidance.

Computed tomography (CT) is a major medical imaging modality. Clinical CT scenarios, such as low-dose screening, sparse-view scanning, and metal implants, often lead to severe noise and artifacts in reconstructed images, requiring improved reconstruction techniques. The introduction of deep learning has significantly advanced CT image reconstruction. However, obtaining paired training data remains rather challenging due to patient motion and other constraints. Although deep learning methods can still perform well with approximately paired data, they inherently carry the risk of hallucination due to data inconsistencies and model instability. In this paper, we integrate the data fidelity with the state-of-the-art generative AI model, referred to as the Poisson flow generative model (PFGM) with a generalized version PFGM++, and propose a novel CT framework: Flow-Oriented Reconstruction Conditioning Engine (FORCE). In our experiments, the proposed method shows superior performance in various CT imaging tasks, outperforming existing unsupervised reconstruction approaches.

X-ray photon-counting computed tomography (PCCT) for extremity allows multi-energy high-resolution (HR) imaging but its radiation dose can be further improved. Despite the great potential of deep learning techniques, their application in HR volumetric PCCT reconstruction has been challenged by the large memory burden, training data scarcity, and domain gap issues. In this paper, we propose a deep learning-based approach for PCCT image reconstruction at halved dose and doubled speed for a New Zealand clinical trial. Specifically, we design a patch-based volumetric refinement network to alleviate the GPU memory limitation, train network with synthetic data, and use model-based iterative refinement to bridge the gap between synthetic and clinical data. Our results in a reader study of 8 patients from the clinical trial demonstrate a great potential to cut the radiation dose to half that of the clinical PCCT standard without compromising image quality and diagnostic value.

Photon-counting computed tomography (PCCT) has emerged as a transformative imaging modality, enabling enhanced spatial resolution, and multi-energy acquisition with energy-discriminating detectors of significantly smaller detector elements. However, both energy discriminating power and reduced detector pixel size result in fewer detected photons per measurement, inherently increasing noise in reconstructed images. In this study, we propose ZS4D, a zero-shot self-similarity-steered denoiser for PCCT reconstruction. Specifically, a self-similarity denoiser is pre-trained in a self-supervised manner by leveraging spectral correlations through multi-energy extraction and capturing volumetric context via the complementary synergy of axial and sagittal planes. The pre-trained denoiser is then integrated as a prior into an iterative reconstruction framework, enabling effective noise suppression and structural preservation. Extensive experiments demonstrate that ZS4D adapts well to varying noise levels and significantly enhances image quality in both simulated and pre-clinical PCCT datasets. Also, ZS4D demonstrates effectiveness in deblurring tasks. Furthermore, our denoiser pretrained on clinical PCCT data is shown to enhance the spatial resolution of conventional CT images.

Foundation models (FMs) are rapidly reshaping medical imaging, shifting the field from narrowly trained, task-specific networks toward large, general-purpose models that can be adapted across modalities, anatomies, and clinical tasks. In this review, we synthesize the emerging landscape of medical imaging FMs along three major axes: principles of FM design, applications of FMs, and forward-looking challenges and opportunities. Taken together, this review provides a technically grounded, clinically aware, and future-facing roadmap for developing FMs that are not only powerful and versatile but also trustworthy and ready for responsible translation into clinical practice.

Self-supervised learning (SSL) has emerged as a crucial technique in image processing, encoding, and understanding, especially for developing today’s vision foundation models that utilize large-scale datasets without annotations to enhance various downstream tasks. This study introduces a novel SSL approach, Information-Maximized Soft Variable Discretization (IMSVD), for image representation learning. Specifically, IMSVD softly discretizes each variable in the latent space, enabling the estimation of their probability distributions over training batches and allowing the learning process to be directly guided by information measures. Motivated by the MultiView assumption, we propose an information-theoretic objective function to learn transform-invariant, non-trivial, and redundancy-minimized representation features. We then derive a joint-cross entropy loss function for self-supervised image representation learning, which theoretically enjoys superiority over the existing methods in reducing feature redundancy. Notably, our non-contrastive IMSVD method statistically performs contrastive learning. Extensive experimental results demonstrate the effectiveness of IMSVD on various downstream tasks in terms of both accuracy and efficiency. Thanks to our variable discretization, the embedding features optimized by IMSVD offer unique explainability at the variable level. IMSVD has the potential to be adapted to other learning paradigms. Our code is publicly available at https://github.com/niuchuangnn/IMSVD.

Deep learning methods have impacted almost every research field, demonstrating notable successes in medical imaging tasks such as denoising and super-resolution. However, the prerequisite for deep learning is data at scale, but data sharing is expensive yet at risk of privacy leakage. As cutting-edge AI generative models, diffusion models have now become dominant because of their rigorous foundation and unprecedented outcomes. Here we propose a latent diffusion approach for data synthesis without compromising patient privacy. In our exemplary case studies, we develop a latent diffusion model to generate medical CT, MRI, and PET images using publicly available datasets. We demonstrate that state-of-the-art deep learning-based denoising/super-resolution networks can be trained on our synthetic data to achieve image quality with no significant difference from what the same network can achieve after being trained on the original data. In our advanced diffusion model, we specifically embed a safeguard mechanism to protect patient privacy effectively and efficiently. Our approach allows privacy-proof public sharing of diverse big datasets for development of deep models, potentially enabling federated learning at the level of input data instead of local network weights.

Low-dose computed tomography (LDCT) reduces radiation exposure but often degrades image quality, potentially compromising diagnostic accuracy. Existing deep learning-based denoising methods focus primarily on pixel-level mappings, overlooking the potential benefits of high-level semantic guidance. Recent advances in vision-language models (VLMs) suggest that language can serve as a powerful tool for capturing structured semantic information, offering new opportunities to improve LDCT reconstruction. In this paper, we introduce LangMamba, a Language-driven Mamba framework for LDCT denoising that leverages VLM-derived representations to enhance supervision from normal-dose CT (NDCT). LangMamba follows a two-stage learning strategy. First, we pre-train a Language-guided AutoEncoder (LangAE) that leverages frozen VLMs to map NDCT images into a semantic space enriched with anatomical information. Second, we synergize LangAE with two key components to guide LDCT denoising: Semantic-Enhanced Efficient Denoiser (SEED), which enhances NDCT-relevant local semantic while capturing global features with efficient Mamba mechanism, and Language-engaged Dual-space Alignment (LangDA) Loss, which ensures that denoised images align with NDCT in both perceptual and semantic spaces. Extensive experiments on two public datasets demonstrate that LangMamba outperforms conventional state-of-the-art methods, significantly improving detail preservation and visual fidelity. Remarkably, LangAE exhibits strong generalizability to unseen datasets, thereby reducing training costs. Furthermore, LangDA loss improves explainability by integrating language-guided insights into image reconstruction and offers a plug-and-play fashion. Our findings shed new light on the potential of language as a supervisory signal to advance LDCT denoising. The code is publicly available on https://github.com/hao1635/LangMamba.

In computed tomography (CT), the presence of metal parts in the scanned region results in metal artifacts in the reconstructed images, which can significantly impact diagnosis and treatment planning. Consequently, removing metal artifacts has been a critical area of research in clinical practice. In this paper, we propose a metal artifact reduction (MAR) algorithm based on dual-domain denoising diffusion probabilistic models (DDPM). Our approach begins with pre-processing with linear interpolation (LI) and refinement with a convolutional neural network (CNN) to generate an initial reprojection. Then, two DDPM networks are employed: one to synthesize the corrupted sinogram and the other to optimize the resultant images in the image domain. The experimental results show that our algorithm utilizes two specialized DDPMs and achieves superior performance. The sinogram-domain DDPM reconstructs a high-quality sinogram, while the image-domain DDPM effectively removes remaining artifacts. Synergistically, these contributions lead to a significant improvement in overall image quality. Furthermore, our method successfully addresses the hallucination issues observed in the generic DDPM, enhancing the applicability of DDPM in medical imaging.