Jinshan Pan

Medical image restoration (MedIR) involves heterogeneous modalities with distinct imaging characteristics, degradation processes, and restoration objectives. Learning a unified model is challenging because shared representations must accommodate modality-dependent structures and task-specific degradations. We propose DRSCP-Net, an all-in-one MedIR framework coupling a degradation-aware routing network with structure-aware cross-modal priors. The framework follows two training stages. First, Structure-Aware Vision–Language Adaptation (SAVLA) extends Degradation-Aware CLIP with edge-aware structural modeling and uses medical image–text contrastive learning to obtain structure-sensitive degradation representations. Second, a Degradation-Aware Routing Instruction Mechanism (DARIM) combines image features with a learnable instruction dictionary to generate expert-selection instructions, while deriving global and layer-specific prompts from the image features. Within Prompt-Guided Expert Selection Transformer (PGES-Former) blocks, routing instructions determine expert activation, and degradation representations from SAVLA are incorporated through cross-attention followed by hierarchical prompt-conditioned gating. At the bottleneck, Context-Gated Dual-Domain Modulation (CGDM) integrates spatial and frequency-domain features under global prompt guidance, whereas Adaptive Gated Fusion (AGF) controls feature transfer across encoder–decoder skip connections. No text input is required during restoration. Experiments on MRI super-resolution, CT denoising, and PET synthesis demonstrate competitive performance in both task-specific and all-in-one settings. In the all-in-one setting, DRSCP-Net achieves an average PSNR of 34.37 dB, tying for the highest among the compared methods, and obtains the best or tied-best results on three of the six task-wise metrics. Joint training yields positive transfer on all three tasks, and downstream segmentation exhibits improved average agreement with reference masks generated from high-quality images. Code will be made available at https://github.com/fanamber831/DRSCP-Net}{\textcolor{red}{https://github.com/fanamber831/DRSCP-Net.

Benefiting from the effectiveness of the self-attention mechanisms in the Transformer framework for modeling non-local features of images, significant progress has been achieved in image super-resolution. We note that existing self-attention mechanisms usually explore all similarities of the tokens between the queries and keys for the feature aggregation. However, using all the similarities does not effectively facilitate the high-quality image reconstruction as not all the tokens from the queries are relevant to those in keys. We further note that self-attention mechanisms are less effective for local feature exploration, which are less effective for the structural detail restoration. To overcome these problems, we develop a simple yet effective adaptive sparse self-attention method to utilize the most useful information of tokens for image restoration. We first develop a local spatial-variant feature estimation method to build the query and key used in the self-attention so that local information can be better modeled. Then, we present a simple yet effective sparse self-attention to adaptively select the most useful similarity values from the self-attention matrix for better the feature aggregation. We analyze that the proposed method models both local and non-local features and thus facilitates better structural detail restoration. We further show that the proposed method can serve as an alternative to existing self-attention mechanisms for better image restoration. Experimental results show that the proposed method performs favorably against state-of-the-art ones on benchmark datasets in terms of accuracy and model complexity.

The key success of existing video super-resolution (VSR) methods stems mainly from exploring spatial and temporal information that is usually achieved by a temporal propagation with alignment strategies. However, inaccurate alignment usually leads to significant artifacts that will be accumulated during propagation and thus affect video restoration. Moreover, only propagating the same timestep features forward or backward does not handle the videos with complex motion or occlusion. To address these issues, we propose a collaborative feedback discriminative (CFD) method to correct inaccurate aligned features and better model spatial and temporal information for VSR. Specifically, we first develop a discriminative alignment correction (DAC) method to reduce the influences of the artifacts caused by inaccurate alignment. Then, we propose a collaborative feedback propagation (CFP) module based on feedback and gating mechanisms to explore spatial and temporal information of different timestep features from forward and backward propagation simultaneously. Finally, we embed the proposed DAC and CFP into commonly used VSR networks to verify the effectiveness of our method. Experimental results demonstrate that our method improves the performance of existing VSR models while maintaining a lower model complexity.

Image deraining plays a pivotal role in low-level computer vision, serving as a prerequisite for robust outdoor surveillance and autonomous driving systems. While deep learning paradigms have achieved remarkable success in firmly aligned settings, they often suffer from severe performance degradation when generalized to unseen Out-of-Distribution (OOD) scenarios. This failure stems primarily from the significant domain discrepancy between synthetic training datasets and the complex physical dynamics of real-world rain. To address these challenges, this paper proposes a pioneering cross-scenario deraining adaptation framework. Diverging from conventional approaches, our method obviates the requirements for paired rainy observations in the target domain, leveraging exclusively rain-free background images. We design a Superpixel Generation (Sup-Gen) module to extract stable structural priors from the source domain using Simple Linear Iterative Clustering. Subsequently, a Resolution-adaptive Fusion strategy is introduced to align these source structures with target backgrounds through texture similarity, ensuring the synthesis of diverse and realistic pseudo-data. Finally, we implement a pseudo-label re-Synthesize mechanism that employs multi-stage noise generation to simulate realistic rain streaks. This framework functions as a versatile plug-and-play module capable of seamless integration into arbitrary deraining architectures. Extensive experiments on state-of-the-art models demonstrate that our approach yields remarkable PSNR gains of up to 32% to 59% in OOD domains while significantly accelerating training convergence.

Removing local blur caused by moving objects is challenging, especially in low signal-to-noise ratio environments, as the moving objects are usually significantly blurry while the static background remains relatively clear. Existing methods relying on local blur detection often suffer from inaccuracies and cannot generate satisfactory results when focusing solely on blurred regions. In this paper, we present an effective diffusion model guided with long-exposure and short-exposure images for realistic local motion deblurring. Specifically, we first propose a context-based local blur detection module. Different from existing methods that rely on pixel-wise classification, we leverage contextual information to generate semantically coherent blur region identification, which preserves the integrity of blur regions. Then, we present a blurry-aware guided image restoration method to remove local blur by discriminately handling the blurry and clear regions. Finally, a structure-guided diffusion model is developed to achieve realistic image restoration by exploring useful information from the above blurry-aware guided image restoration result and the short-exposure image as guidance. The proposed method, ExpDiff, is trained in an end-to-end manner. Our extensive experimental results show that the proposed ExpDiff performs favorably against state-of-the-art methods.