Huidong Xie

In medical image analysis, it is common to collect multiple annotations from different clinical experts or raters to mitigate potential diagnostic inaccuracies. However, in computer vision research, it has been conventional to adopt ground-truth labels generated either by majority voting or by selecting a single rater's annotation. Such practices overlook the rich information embedded in the original multi-rater annotations, particularly the patterns of agreement and disagreement among experts. In this paper, we propose to explicitly model multi-rater (dis-)agreement, referred to as MRNet+, which offers several key innovations. First, an Expertise-aware Inferring Module (EIM) is developed to incorporate the expertise level of each rater as prior knowledge, thereby forming discriminative semantic features. Second, our method efficiently infers interobserver variability through a disentangled erasing-based Multi-rater Knowledge Modeling (MKM) module. Third, the cues pertaining to multi-rater (dis-)agreement are further leveraged through a Multi-rater Perception Module (MPM) to enhance segmentation performance. To the best of our knowledge, this is the first method to generate calibrated predictions that account for varying expertise levels in medical image segmentation. Moreover, MRNet+ is computationally efficient, achieving real-time performance with a speed of 89 FPS on 256×256 images. We validate our framework across ten medical segmentation tasks spanning diverse imaging modalities. In all cases, MRNet+ consistently outperforms state-of-the-art specialized methods, highlighting its effectiveness and broad applicability to real-world medical image segmentation scenarios. Source code is available at https://github.com/jiwei0921/MRNet_plus.

Quantitative ¹⁸F-FDG Patlak K_i images provide physiologically meaningful measures of glucose influx, but they are substantially noisier than SUV images because they are estimated from short-duration dynamic frames. This study aims to improve the quality and quantitative reliability of K_i images by applying a self-supervised deep learning framework for dynamic PET denoising. We propose the Patlak-Guided Self-Supervised Denoising Network (PG-SSDNet), a multi-frame CycleGAN-based model that incorporates a Patlak consistency loss to preserve tracer kinetics across frames. PG-SSDNet was evaluated on 14-frame and 6-frame whole-body ¹⁸F-FDG dynamic PET datasets from 27 patients. Performance was compared with a supervised U-Net, a single-frame CycleGAN, and a multi-frame CycleGAN without Patlak guidance. Metrics included mean normalized residual sum of squares (MNRSS) of Patlak fits, liver normalized standard deviation (NSTD), and lesion signal-to-noise ratio (SNR). PG-SSDNet consistently reduced noise while maintaining kinetic fidelity. It achieved the lowest Patlak fitting errors (MNRSS: 0.98% ± 0.14% for 14 frames, 0.85% ± 0.11% for 6 frames), the lowest liver NSTD values (0.105 ± 0.037 for 14 frames, 0.211 ± 0.107 for 6 frames), and the highest lesion SNR across all methods, while preserving accurate mean Ki values. PG-SSDNet effectively denoises ¹⁸F-FDG dynamic PET data while preserving Patlak linearity, improving both lesion quantification and image quality. Unlike supervised methods, it requires no paired noisy/clean datasets, making it practical for clinical deployment. Future work will extend PG-SSDNet to other tracers with different kinetic properties and to larger multi-center datasets.

Deep learning-based positron emission tomography (PET) image denoising offers the potential to reduce radiation exposure and scanning time by transforming low-count images into high-count equivalents. However, existing methods typically blur crucial details, leading to inaccurate lesion quantification. This paper proposes a lesion-perceived and quantification-consistent modulation (LeqMod) strategy for enhanced PET image denoising, via employing downstream lesion quantification analysis as auxiliary tools. The LeqMod is a plug-and-play design adaptable to a wide range of model architectures, modulating the sampling and optimization procedures of model training without adding any computational burden to the inference phase. Specifically, the LeqMod consists of two components, the lesion-perceived modulation (LeMod) and the multiscale quantification-consistent modulation (QuMod). The LeMod enhances lesion contrast and visibility by allocating higher sampling weights and stricter loss criteria to lesion-present samples determined by an auxiliary segmentation network than lesion-absent ones. The QuMod further emphasizes quantification accuracy for both the mean and maximum standardized uptake value (SUV_mean and SUV_max) across multiscale sub-regions throughout the entire image, thereby reducing biases of denoised results relative to high-count references. Experiments conducted on large PET datasets from multiple centers and vendors, and varying noise levels demonstrated the LeqMod efficacy across various denoising frameworks. Compared to frameworks without LeqMod, the integration of LeqMod reduces the lesion SUVmax bias by 5.92% on average and increases the peak signal-to-noise ratio (PSNR) by 0.36 on average, when denoising images across participating sites. (Code is available at https://github.com/mhxiaaa/LeqModPETdenoising).

Low-dose computed tomography (LDCT) reduces radiation exposure but often degrades image quality, potentially compromising diagnostic accuracy. Existing deep learning-based denoising methods focus primarily on pixel-level mappings, overlooking the potential benefits of high-level semantic guidance. Recent advances in vision-language models (VLMs) suggest that language can serve as a powerful tool for capturing structured semantic information, offering new opportunities to improve LDCT reconstruction. In this paper, we introduce LangMamba, a Language-driven Mamba framework for LDCT denoising that leverages VLM-derived representations to enhance supervision from normal-dose CT (NDCT). LangMamba follows a two-stage learning strategy. First, we pre-train a Language-guided AutoEncoder (LangAE) that leverages frozen VLMs to map NDCT images into a semantic space enriched with anatomical information. Second, we synergize LangAE with two key components to guide LDCT denoising: Semantic-Enhanced Efficient Denoiser (SEED), which enhances NDCT-relevant local semantic while capturing global features with efficient Mamba mechanism, and Language-engaged Dual-space Alignment (LangDA) Loss, which ensures that denoised images align with NDCT in both perceptual and semantic spaces. Extensive experiments on two public datasets demonstrate that LangMamba outperforms conventional state-of-the-art methods, significantly improving detail preservation and visual fidelity. Remarkably, LangAE exhibits strong generalizability to unseen datasets, thereby reducing training costs. Furthermore, LangDA loss improves explainability by integrating language-guided insights into image reconstruction and offers a plug-and-play fashion. Our findings shed new light on the potential of language as a supervisory signal to advance LDCT denoising. The code is publicly available on https://github.com/hao1635/LangMamba.

Positron Emission Tomography (PET) is an important clinical imaging tool but inevitably introduces radiation exposure to patients and healthcare providers. Reducing the tracer injection dose and eliminating the CT acquisition for attenuation correction can reduce the overall radiation dose, but often results in PET with high noise and bias. Thus, it is desirable to develop 3D methods to translate the non-attenuation-corrected low-dose PET (NAC-LDPET) into attenuation-corrected standard-dose PET (AC-SDPET). Recently, diffusion models have emerged as a new state-of-the-art deep learning method for image-to-image translation, better than traditional CNN-based methods. However, due to the high computation cost and memory burden, it is largely limited to 2D applications. To address these challenges, we developed a novel 2.5D Multi-view Averaging Diffusion Model (MADM) for 3D image-to-image translation with application on NAC-LDPET to AC-SDPET translation. Specifically, MADM employs separate diffusion models for axial, coronal, and sagittal views, whose outputs are averaged in each sampling step to ensure the 3D generation quality from multiple views. To accelerate the 3D sampling process, we also proposed a strategy to use the CNN-based 3D generation as a prior for the diffusion model. Our experimental results on human patient studies suggested that MADM can generate high-quality 3D translation images, outperforming previous CNN-based and Diffusion-based baseline methods. The code is available at https://github.com/tianqic/MADM.

Attenuation correction (AC) is a critical step to ensure accurate quantitative PET imaging. To eliminate the radiation dose from CT, deep learning (DL)-based methods have been extensively investigated to generate the CT-equivalent attenuation map (μ-CT) directly from the PET signal. However, almost all previous studies only focus on 18F-FDG due to its extensive data availability which is suitable for DL model training. For other less common tracer types, it is generally believed that new models must be trained separately on these tracer-specific data to ensure reasonable performance. In this work, we explored the cross-tracer generalizability of μ-DL generation DL models -primarily focusing on whether a model trained on a commonly used tracer like 18F-FDG can be effectively applied to less common tracers such as 68Ga-DOTATE and 18F-Fluciclovine, and vice versa. Unlike methods that directly generate attenuation-corrected (AC) PET images from non-attenuation corrected (NAC) PET images or MLAA reconstructions, we generate the CT-based deep learning attenuation maps (μ-DL) using MLAA reconstruction with the combined input of attenuation maps (μ-MLAA) and tracer activity (λ-MLAA). This μ-DL is then used for attenuation correction to obtain the final AC PET image. Our comprehensive evaluations on both μ-CT generation and the PET reconstruction found that the DL model trained on one specific tracer can be adapted to other tracers with competitive performance when compared to the tracer-specific trained DL model. The 18F-FDG-trained model demonstrated the best generalizability when applied to less common tracer types which often have relatively fewer available data for training. Additionally, we investigated the role of the μ-MLAA and λ-MLAA as inputs for the network performance. We found that combining both inputs resulted in the best performance, but the μ-MLAA contributed more significantly compared to the λ-MLAA.

Anatomically guided PET reconstruction using MRI information has been shown to have the potential to improve PET image quality. However, these improvements are limited to PET scans with paired MRI information. In this work we employed a diffusion probabilistic model (DPM) to infer T1-weighted-MRI (deep-MRI) images from FDG-PET brain images. We then use the DPM-generated T1w-MRI to guide the PET reconstruction. The model was trained with brain FDG scans, and tested in datasets containing multiple levels of counts. Deep-MRI images appeared somewhat degraded and in some cases showed inaccuracies compared to the acquired MRI images. Regarding PET image quality, volume of interest analysis in different brain regions showed that both PET reconstructed images using the acquired and the deep-MRI images improved image quality compared to OSEM. Same conclusions were found analysing the decimated datasets. A subjective evaluation performed by two physicians confirmed that OSEM scored consistently worse than the MRI-guided PET images and no significant differences were observed between the MRI-guided PET images. This proof of concept shows that it is possible to infer DPM-based MRI imagery to guide the PET reconstruction, enabling the possibility of changing reconstruction parameters such as the strength of the prior on anatomically guided PET reconstruction in the absence of MRI.