Yan Zhang

Temporal information, as an inherent attribute of video sequences, plays a crucial role in visual object tracking. It provides implicit cues to capture dynamic changes and invariant characteristics of both the tracking target and the environment. Adequately leveraging it enables tracking models to enhance their adaptability, robustness, and tracking performance. However, existing tracking methods often suffer from redundancy in temporal features, inadequate temporal context propagation, and reliance on empirical template updates, thus leading to suboptimal utilization of such cues. To address these limitations, this paper proposes M-STEP, a hierarchical visual Transformer tracking method that incorporates the shallow feature extraction (SFE) encoder, multi-scale template enhancement (MSTE) encoder, spatiotemporal information fusion (SIF) encoder, and temporal propagation enhancement (TPE) module into a unified framework to effectively enhance and propagate the multi-scale template and spatiotemporal information. Specifically, the SFE encoder employs a simple yet efficient MLP block to effectively preserve shallow spatial details while maintaining linear computational complexity. Second, the MSTE encoder utilizes a multi-scale pyramid pooling mechanism to capture the invariant target features under complex appearance and environmental changes, thereby maintaining the target-aware discriminative ability. Then, the TPE module combines Mamba and attention operations to fuse multi-scale temporal features, aiming to adaptively strengthen temporal correlations, reduce temporal redundancy, and ensure efficient propagation of temporal information. Finally, the SIF encoder integrates multi-scale spatiotemporal information to further improve the model's ability to capture motion and appearance diversity. Experimental results on eight benchmark datasets demonstrate that M-STEP achieves state-of-the-art performance, highlighting the critical role of multi-scale template aggregation and spatiotemporal information propagation in advancing tracking performance.

Prevailing pan-sharpening methods tackle the ill-posed challenge of reconstructing high-resolution multispectral (HRMS) images from low-resolution multispectral (LRMS) and panchromatic (PAN) inputs. This ill-posed nature introduces distortions such as blurred spatial edges and spectral color deviations, which compromise the fidelity of the reconstructed image. To address this, we propose S²CMamba, a dual-branch framework that leverages Mamba’s efficient contextual modeling and enforces spatial and spectral consistency through tailored priors, effectively alleviating the ill-posed nature of the pan-sharpening. The spatial context branch utilizes the Windowed Spatial Local Mamba (WSLM) for local details and the Global Spatial Interaction Mamba (GSIM) for long-range structures. Within WSLM, a Manifold Preservation (MP) constraint is proposed to align HRMS features with the low-dimensional manifold consistency of PAN and LRMS, thereby mitigating high-dimensional distortions and enhancing spatial consistency. Meanwhile, the spectral branch integrates multi-scale feature extraction and designed Spectral Context Mamba (SCM) to capture spectral context. Moreover, the spatial and spectral properties of multispectral images are investigated, and the wavelet transforms are introduced for better accomplish consistency. By incorporating contextual information, S²CMamba reduces the ambiguity of the solution space, while wavelet-based consistency constraints and designed MP prior further alleviate the ill-posed nature. Extensive experiments on benchmark datasets demonstrate that S²CMamba surpasses state-of-the-art methods, validating the efficacy of this approach in addressing the ill-posed nature of pan-sharpening tasks.

Current deepfake detection methods commonly use data augmentation and authenticity-content disentanglement to extract more generalized features for detection tasks. However, these methods rely exclusively on low-level spatial artifacts to distinguish real from fake images, which presents significant challenges in accurately capturing the rich forgery cues. Deepfakes create discrepancies between forged and original facial features within the face-recognition (FR) embedding space, which can serve as an additional cue for detection. To better exploit the artifacts in deepfake images, we propose a novel detection method that enhances the detector’s perception capability by incorporating not only the real and fake samples during training, but also the visual residual between real and fake images. Meanwhile, we integrate the discrepancy in facial embedding between the real and fake samples into the training procedure of artifact extraction, serving as a guidance signal with strong knowledge provided by the pretrained face recognition model. Specialized distillation loss along with additional cross-entropy losses are designed to enhance the detection capability. Experiments on multiple benchmarks demonstrate the superiority of the proposed approach in deepfake detection over literature methods.

Remote Sensing (RS) image deblurring and Super-Resolution (SR) are common tasks in computer vision that aim at restoring RS image detail and spatial scale, respectively. However, real-world RS images often suffer from a complex combination of global low-resolution (LR) degeneration and local blurring degeneration. Although carefully designed deblurring and SR models perform well on these two tasks individually, a unified model that performs jointly RS image deblurring and super-resolution (JRSIDSR) task is still challenging due to the vital dilemma of reconstructing the global and local degeneration simultaneously. Additionally, existing methods struggle to capture the interrelationship between deblurring and SR processes, leading to suboptimal results. To tackle these issues, we give a unified theoretical analysis of RS images’ spatial and blur degeneration processes and propose a dual-branch parallel network named AKMD-Net for the JRSIDSR task. AKMD-Net consists of two main branches: deblurring and super-resolution branches. In the deblurring branch, we design a pixel-adjustable kernel block (PAKB) to estimate the local and spatial-varying blur kernels. In the SR branch, a multi-domain attention block (MDAB) is proposed to capture the global contextual information enhanced with high-frequency details. Furthermore, we develop an adaptive feature fusion (AFF) module to model the contextual relationships between the deblurring and SR branches. Finally, we design an adaptive Wiener loss (AW Loss) to depress the prior noise in the reconstructed images. Extensive experiments demonstrate that the proposed AKMD-Net achieves state-of-the-art (SOTA) quantitative and qualitative performance on commonly used RS image datasets. The source code is publicly available at https://github.com/zpc456/AKMD-Net .

Remote Sensing (RS) image deblurring and Super-Resolution (SR) are common tasks in computer vision that aim at restoring RS image detail and spatial scale, respectively. However, real-world RS images often suffer from a complex combination of global low-resolution (LR) degeneration and local blurring degeneration. Although carefully designed deblurring and SR models perform well on these two tasks individually, a unified model that performs jointly RS image deblurring and super-resolution (JRSIDSR) task is still challenging due to the vital dilemma of reconstructing the global and local degeneration simultaneously. Additionally, existing methods struggle to capture the interrelationship between deblurring and SR processes, leading to suboptimal results. To tackle these issues, we give a unified theoretical analysis of RS images' spatial and blur degeneration processes and propose a dual-branch parallel network named AKMD-Net for the JRSIDSR task. AKMD-Net consists of two main branches: deblurring and super-resolution branches. In the deblurring branch, we design a pixel-adjustable kernel block (PAKB) to estimate the local and spatial-varying blur kernels. In the SR branch, a multi-domain attention block (MDAB) is proposed to capture the global contextual information enhanced with high-frequency details. Furthermore, we develop an adaptive feature fusion (AFF) module to model the contextual relationships between the deblurring and SR branches. Finally, we design an adaptive Wiener loss (AW Loss) to depress the prior noise in the reconstructed images. Extensive experiments demonstrate that the proposed AKMD-Net achieves state-of-the-art (SOTA) quantitative and qualitative performance on commonly used RS image datasets. The source code is publicly available at https://github.com/zpc456/AKMD-Net.

Face super-resolution (FSR) is critical for bolstering intelligent security in Internet of Things (IoT) systems. Recent deep learning-driven FSR algorithms have attained remarkable progress. However, they always require separate model training and optimization for each scaling factor or input resolution, leading to inefficiency and impracticality. To overcome these limitations, we propose SAFNet, an innovative framework tailored for scale-adaptive FSR with arbitrary input resolution. SAFNet integrates scale information into representation learning to enable adaptive feature extraction and introduces dual-embedding attention to boost adaptive feature reconstruction. It leverages facial self-similarity and spatial-frequency collaboration to achieve precise scale-aware SR representations. This is attained through three key modules: 1) the scale adaption guidance unit (SAGU); 2) the scale-aware nonlocal self-similarity (SNLS) module; and 3) the spatial-frequency interactive modulation (SFIM) module. SAGU imports scaling factors using frequency encoding, SNLS exploits self-similarity to enrich feature representations, and SFIM incorporates spatial and frequency information to predict target pixel values adaptively. Comprehensive evaluations across four benchmark datasets reveal that SAFNet outperforms the second-best compared state-of-the-art (SOTA) method by about 0.2 dB/0.007 in PSNR/SSIM ( $\times 4$ on CelebA) with reduced 18.68%/42.64% computational complexity/time cost. This demonstrates SAFNet’s effectiveness and superiority, showcasing its potential as a promising solution for scale and input resolution adaptation challenges in FSR. The code will be available at https://github.com/ICVIPLab/SAFNet.

Recent CNN-driven face super-resolution (FSR) technologies have achieved excellent breakthroughs by incorporating facial prior knowledge. However, most of them suffer from some obvious limitations. They always estimate facial priors from input low-resolution (LR) faces or coarsely enhanced LR faces, obtaining unfaithful priors that cannot be adequately exploited. This may bring noticeable artifacts to the target results, especially for large scaling factors, deteriorating the fidelity and naturalness and generating suboptimal reconstructed results. In this paper, we propose a two-stage prior-guided FSR approach to learn facial prior knowledge from the optimal SR results of stage one and explore the complementarity between priors to further guide more accurate reconstruction in stage two. Specifically, we develop an efficient local and global interactive hybrid network incorporating facial semantic and geometric priors for more discriminative results. To reach this, we devise a multiscale interconnected symmetric encoder-decoder architecture composed of Prior Interaction-Integration Modules (PIIMs), the Coarse-to-fine Feature Refinement Module (CFRM), and Feature Aggregation Modulation Modules (FAMMs). The encoder concentrates on hierarchically extracting multiscale features. The CFRM is devised to explore the potential correlations between the encoder and the decoder and further guide the refinement and reinforcement of the encoded features. The decoder aims to take full advantage of informative multiscale encoded features to reconstruct high-quality SR representations. Comprehensive evaluation and visualization results on four benchmark datasets demonstrate the superiority of the proposed PLGNet over current state-of-the-art methods. The source code of PLGNet will be available at https://github.com/lil808/PLGNet.git.