Lihe Zhang

With the advent of foundation models, fine-tuning has become a prominent approach for long-tailed learning. Recent methods primarily explore different long-tailed strategies under the standard fine-tuning paradigm. However, this default paradigm typically discards the pre-trained classifier and instead re-trains a new one from scratch. In this paper, we observe that such re-trained classifiers suffer from weakened discriminative ability and semantic awareness under tail-data scarcity, exhibiting severe imbalance in class-discriminative channels and mislearning general features for tail classes. In contrast, the overlooked pre-trained classifier, shaped by large-scale pre-training, behaves much closer to the oracle. Motivated by this, we propose a simple yet effective fine-tuning paradigm, PTClf (Pre-Trained Classifier helps), which leverages priors from the pre-trained classifier to guide the re-trained one in learning tail classes. Experiments show that PTClf delivers remarkable benefits for long-tailed data, especially on tail classes, while also exhibiting strong versatility in few-shot learning and domain generalization. The code is available at https://github.com/ckshang/PTClf.

Accurate medical image segmentation is critical for early medical diagnosis. Most existing methods are based on U-shape structure and use element-wise addition or concatenation to fuse different level features progressively in decoder. However, both the two operations easily generate plenty of redundant information, which will weaken the complementarity between different level features, resulting in inaccurate localization and blurred edges of lesions. To address this challenge, we propose a general multi-scale in multi-scale subtraction network (M ² SNet) to finish diverse segmentation from medical image. Specifically, we first design a basic subtraction unit (SU) to produce the difference features between adjacent levels in encoder. Next, we expand the single-scale SU to the intra-layer multi-scale SU, which can provide the decoder with both pixel-level and structure-level difference information. Then, we pyramidally equip the multi-scale SUs at different levels with varying receptive fields, thereby achieving the inter-layer multi-scale feature aggregation and obtaining rich multi-scale difference information. In addition, we build a training-free network “LossNet” to comprehensively supervise the task-aware features from bottom layer to top layer, which drives our multi-scale subtraction network to capture the detailed and structural cues simultaneously. Without bells and whistles, our method performs favorably against most state-of-the-art methods under different evaluation metrics on eleven datasets of four different medical image segmentation tasks of diverse image modalities, including color colonoscopy imaging, ultrasound imaging, computed tomography (CT), and optical coherence tomography (OCT). The source code can be available at https://github.com/Xiaoqi-Zhao-DLUT/MSNet .

Reconstruction-based methods have become predominant in unsupervised anomaly detection, due to the powerful reconstruction capabilities provided by diffusion models. However, under a multi-class setting, diffusion-based methods are prone to two types of reconstruction hallucinations: local artifacts due to noise-degraded textures and mismatches caused by inter-class feature drift, leading to inaccurate detection results. In this paper, we propose a Dual-Guidance Diffusion Model (DGDM) for anomaly detection, which enhances the reconstruction capabilities of diffusion models in multi-class anomaly detection by calibration and classification. Discrepancy-aware Prototypical Guidance Module (DPGM) calibrates the reconstructed differential regions and utilizes routed vector quantization to preserve the input image’s normal patterns while inverting anomaly patterns. This process generates a refined prototype prompt that guides the reconstruction of local details in texture-rich normal regions, which are particularly challenging for reconstruction. Class-aware Guidance Module (CAGM) provides the denoising model’s encoder with classification capabilities, ensuring that distinct classes are well-separated in representation space. This facilitates the generation of accurate class-aware prompts to further guide the denoising model’s decoder for image reconstruction. The dual guidance promotes the diffusion model to consider multi-class anomalies from both global and local perspectives. Extensive experiments conducted across five diverse public datasets using eight evaluation metrics demonstrate that the proposed method outperforms existing approaches in overall performance. Code will be available at https://github.com/yuanzhao-CVLAB/DGDM.

Real-time and accurate portrait matting in videos is a challenging problem in computer vision research. Recent approaches have explored incorporating prior conditions for accurate inference. Notably, some methods ask the user to provide the background image, which requires extra effort from the user to capture the background image and is limited to videos with static backgrounds only. We note that real-time video motion segmentation methods often train a background model to detect the foreground. Our insight of this work is that if we can directly restore the background content from the input video as a prior, we may be able to achieve more precise portrait matting. In addition, this approach could potentially work even with dynamic backgrounds, without requiring additional user input. However, automatically restoring the background content is not straightforward due to the difficulty in distinguishing between foreground and background. While it may seem that stationary pixel values represent the background, these values can vary across frames. To this end, we propose a novel dynamic scene background restoration (DSBR) module that learns a background model by accumulating background content from each input video frame. It restores the current background content, which serves as a matting prior for alpha prediction of the subsequent frame. DSBR is extremely lightweight and can be easily integrated into existing matting models. Based on it, we present a real-time portrait video matting framework, LiveMatte. To more efficiently process high-resolution videos, we also introduce a selective portrait patch enhancement (SPPE) module. Extensive experiments and user studies demonstrate that our method is better and faster than existing methods.

Recent advances in medical multi-modal models focus on specialized image analysis like dermatology, pathology, or radiology. However, they do not fully capture the complexity of real-world clinical diagnostics, which involve heterogeneous inputs and require ongoing contextual understanding during patient-physician interactions. To bridge this gap, we introduce PulseMind, a new family of multi-modal diagnostic models that integrates a systematically curated dataset, a comprehensive evaluation benchmark, and a tailored training framework. Specifically, we first construct a diagnostic dataset, MediScope, which comprises 98,000 real-world multi-turn consultations and 601,500 medical images, spanning over 10 major clinical departments and more than 200 sub-specialties. Then, to better reflect the requirements of real-world clinical diagnosis, we develop the PulseMind Benchmark, a multi-turn diagnostic consultation benchmark with a four-dimensional evaluation protocol comprising proactiveness, accuracy, usefulness, and language quality. Finally, we design a training framework tailored for multi-modal clinical diagnostics, centered around a core component named Comparison-based Reinforcement Policy Optimization (CRPO). Compared to absolute score rewards, CRPO uses relative preference signals from multi-dimensional com-parisons to provide stable and human-aligned training guidance. Extensive experiments demonstrate that PulseMind achieves competitive performance on both the diagnostic consultation benchmark and public medical benchmarks.

In this paper, we present a new dynamic collaborative network for semi-supervised 3D vessel segmentation, termed DiCo. Conventional mean teacher (MT) methods typically employ a static approach, where the roles of the teacher and student models are fixed. However, due to the complexity of 3D vessel data, the teacher model may not always outperform the student model, leading to cognitive biases that can limit performance. To address this issue, we propose a dynamic collaborative network that allows the two models to dynamically switch their teacher-student roles. Additionally, we introduce a multi-view integration module to capture various perspectives of the inputs, mirroring the way doctors conduct medical analysis. We also incorporate adversarial supervision to constrain the shape of the segmented vessels in unlabeled data. In this process, the 3D volume is projected into 2D views to mitigate the impact of label inconsistencies. Experiments demonstrate that our DiCo method sets new state-of-the-art performance on three 3D vessel segmentation benchmarks. The code repository address is https://github.com/xujiaommcome/DiCo

As an essential vision task, infrared small target detection (IRSTD) has seen significant advancements through deep learning. However, critical limitations in current evaluation protocols impede further progress. First, existing methods rely on fragmented pixel- and target-level specific metrics, which fails to provide a comprehensive view of model capabilities. Second, an excessive emphasis on overall performance scores obscures crucial error analysis, which is vital for identifying failure modes and improving real-world system performance. Third, the field predominantly adopts dataset-specific training-testing paradigms, hindering the understanding of model robustness and generalization across diverse infrared scenarios. This paper addresses these issues by introducing a hybrid-level metric incorporating pixel- and target-level performance, proposing a systematic error analysis method, and emphasizing the importance of cross-dataset evaluation. These aim to offer a more thorough and rational hierarchical analysis framework, ultimately fostering the development of more effective and robust IRSTD models. An open-source toolkit has be released to facilitate standardized benchmarking.