Jiantao Zhou

Model fingerprinting, embedding user-specific identifiers (fingerprints) into generated outputs, has recently emerged as a popular solution to protect the intellectual property rights (IPR) of generative text-to-image (T2I) models and prevent unauthorized redistribution. In this work, we reveal a previously unexplored systematic vulnerability in existing generative model fingerprinting methods: they lack robustness against collusion attacks, where multiple attackers combine their models to remove or obscure the fingerprints. To address this issue, we take the first step towards a robust fingerprinting method for T2I models with anti-collusion capabilities. The proposed method encodes strings of bits, namely fingerprints, into the coefficients of a personalized normalization module (PNM) incorporated into T2I models, so that fingerprints can be reliably recovered from any generated image. To defend against collusion attacks and prevent unauthorized model redistribution, we introduce an anti-collusion mechanism based on lossless function-invariant parameter transformations. This mechanism significantly degrades the image generation quality of colluded models, making them effectively unusable. Moreover, our method allows developers to efficiently create multiple copies of fingerprinted T2I models by reparameterizing the PNM without the need for retraining. We also introduce a worst-case optimization strategy to improve robustness against model-level attacks. Our experiments demonstrate that the proposed method achieves high fidelity and robustness across multiple T2I image generation and editing tasks, with fingerprint extraction accuracy exceeding 99.5%. Compared with existing methods, our method demonstrates, for the first time, a notable proactive robustness to collusion attack by significantly increasing the FID of colluded models.

The rapid advancement of AI-generated content (AIGC) presents significant challenges for digital forensics, necessitating robust and generalizable detection frameworks. Existing detection methods primarily rely on visual feature extraction, while vision-language model-based approaches are limited to class-label prompts, failing to capture quality-related artifacts introduced by different generative models. To address this limitation, we introduce QAFD, a novel Quality-Assisted Forgery Detection framework that incorporates image quality information into the detection process. Specifically, we design a quality queried attention block to effectively fuse class-based content prompts with quality-aware text prompts. This integration enhances the model’s ability to capture semantic artifacts related to degradation patterns commonly associated with AI-generated images. Furthermore, we introduce the Quality-Guided Forgery Adapter (QGFA) to incorporate quality-aware textual cues into the visual domain, improving feature extraction for both spatial and frequency-based forgery artifacts. This synergy allows frequency cues to enhance low-level artifact perception, while quality-aware guidance strengthens high-level discriminative representation. Extensive experiments demonstrate that QAFD achieves superior generalization to unseen generative models over three datasets and significantly maintains its robustness against common image post-processing operations. The codes will be released at github.

Single-image reflection removal (SIRR) aims to restore the latent background layer from a reflection-contaminated image. Despite the promising progress achieved by deep learning-based methods, the roles of negative training samples and descriptive prompts for the reflection severity are underexplored in most existing deep SIRR approaches, limiting their reflection removal performance and generalization capability. In this work, we introduce a novel training framework that synergistically leverages learnable prompts and image data to optimize the restoration network. To this end, we define reflection levels corresponding to varying degrees of reflection interference on the background content and learn reflection-level prompts to supervise the SIRR process. We propose an Iterative Reflection Level Reduction (IRLR) framework composed of a Restoration Network Training Module (RNTM) and a Reflection Level Learning Module (RLLM). Specifically, RNTM predicts the background layer under the guidance of prompts learned by RLLM, while RLLM in turn refines these prompts using outputs from RNTM. The two modules are trained iteratively to progressively reduce the reflection levels of estimated background layers. To initialize the prompts, we construct a dedicated reflection-level dataset for pretraining. For adaptively supervising RNTM, we design a new reflection-level-aware strategy to address the challenge of directly aligning the output background with the minimal reflection level. Comprehensive experimental results demonstrate that the proposed method significantly outperforms state-of-the-art methods on average performance across several released datasets, improving PSNR by 0.82 dB and SSIM by 0.0120, respectively. The source code and dataset are available at https://github.com/NamecantbeNULL/IRLR SIRR.

Face-swapping Deepfake technology enables the replacement of one person’s face with another while preserving the facial attributes (e.g., expressions and postures) of the original person being replaced. Although it has various applications, its misuse raises significant security concerns. In recent years, numerous Deepfake forensic techniques have been proposed, relying on binary classification models to detect the authenticity of facial images. However, another important issue, the feasibility of tracing the original face replaced in the deepfake process, has received little attention, as it intuitively seems impossible. In this work, we consider the problem known as the Traceability of Face-swapping Deepfake (TFD). Unlike existing deepfake detection tasks, TFD focuses on restoring the original faces lost in the face-swapping process. Specifically, we reformulate TFD as a specialized image-to-image editing problem and propose a novel diffusion-based framework named FaceReclaim for TFD. To ensure effective traceability, we develop a Multi-Scale Face Attribute Decoupling (FAD) and a Multi-Modal Face Identity Prompting (FIP) for dual-conditioning guidance. FAD extracts multi-scale attribute information decoupled from the synthetic face-swapped images, providing positive guidance to preserve essential attribute features during restoration. FIP integrates both textual and visual instructions, serving as negative guidance to explicitly direct the model in erasing identity traces of the face-swapped image. Our research shows that the synthetic face still retains subtle, traceable features of the original face, enabling its approximation and restoration. We evaluate our model from both visual restoration performance and the face verification perspective. Extensive experiments demonstrate the qualitative and quantitative effectiveness of our scheme.

Learning network representations, also known as network embeddings, has attracted significant attention in recent years. Real-world scenarios often involve networks with multiple views, where each view captures a distinct aspect of the network’s structure. Existing network embedding methods mainly focus on the global information from each view, neglecting the implied relations among multiple views. Additionally, maintaining the scalability of node embeddings while adapting to changes in network topology remains a major challenge. To this end, this article proposes a Cloud-edge Collaboration Network (CC-Net) to learn robust node embeddings in multi-view networks. Specifically, we design a decomposition and regrouping module to capture implied relations within multi-view networks, enabling the generation of comprehensive node representations that integrate information from all sub-networks. Besides, by leveraging the hybrid approach of cloud and edge computing, our proposed CC-Net can efficiently handle the complexities and dynamics of multi-view networks without retraining the entire network. Extensive experiments and analyses on real-world Twitter and YouTube datasets demonstrate the superiority of our approach compared to several benchmark methods, and validate its effectiveness in capturing implied relations and generating robust node embeddings.

With advances in generation techniques, malicious users can easily generate deepfake videos, which can cause severe social problems and trust issues. Therefore, deepfake video detection has received increasing attention in recent years. Given that forgery clues are often subtle and imperceptible, effective detection relies heavily on multi-grained learning. However, existing approaches fail to systematically incorporate multi-grained learning across the key components of network training—namely, the training data, network structure, and supervision strategy—thus limiting their performance. In this article, we propose a multi-grained parallel spatio-temporal deepfake video detection architecture, which introduces a novel framework to mine more discriminative deepfake cues throughout the training pipeline. Firstly, we design a parallel spatio-temporal network combined with a cross-guided mechanism to concurrently extract frame-level spatial features and patch-level temporal features, while leveraging the relationship between spatial artifacts and temporal inconsistencies to enable multi-grained spatio-temporal synchronous learning. Secondly, we propose segment-level data augmentation strategies, including frame-random consistent self-blending and spatio-temporal data augmentation, which improve training data diversity at both frame and patch levels, thereby improving the model’s ability to learn comprehensive deepfake representations. Finally, we construct a multi-grained supervision, comprising a patch-level temporal loss, a distance-based frame-level spatial loss, and a standard segment-level loss, for subtle deepfake feature learning. Extensive experiments demonstrate that our method possesses strong robustness and the generalization ability outperforms the current state-of-the-art methods across a series of deepfake datasets, including FaceForensics++, CelebDF, DFDC, DeeperForensics, and Faceshifter, on average.

Graph representation has become the most efficient method for EEG-based emotion recognition by representing the cognitive functional connectivity between EEG channels as graphs. However, these graph-based methods typically focus only on pairwise relationships and fail to capture multi-way or regional interactions of brain regions. To address the limitation of insufficient topological representation capability in graph and adequately explore the topological information from EEG signals, we propose a novel high-order topological encoder-driven neural network (HOTEN) for explainable EEG emotion recognition, which integrates low-order and high-order EEG features by constructing EEG high-order topology using simplicial complex. To efficiently decode emotional state using low-order and high-order topological features, three encoding modules at different levels are designed, i.e., node-level, edge-level, and triang-lelevel. For pairwise, edge-wise, and region-wise interactions, the graph attention with skip projection (GAS) encoder, edge flow filtering (EFF) encoder, and simplex attention encoder with feature compensation (SAFC) are proposed to extract functional connectivity, edge information flows, and regional topological features, respectively. Additionally, the simplex domain attention is finally introduced to obtain the discriminative features related to emotional responses from three encoders. Such an explainable framework is especially valuable for Smart Consumer Interfaces, where reliable emotion recognition can enable more adaptive, personalized, and human-centric user experiences. Experimental results on two publicly-used EEG datasets demonstrated that the HOTEN model significantly outperforms the popular graph-based methods, indicating the effectiveness of high-order topo-logical features.

Adversarial example detection has emerged as a prominent defense strategy owing to its efficiency in training and deployment. Nevertheless, existing detectors are typically developed under a single-step paradigm, where models become static after training on adversarial examples generated by a single attack. This paradigm is infeasible in dynamic real-world scenarios, since retraining from scratch for each newly encountered attack is impractical and computationally prohibitive. To address this limitation, we propose Continual Adversarial example Detection via Incremental Attack Configuration (IAC-CAD), which pioneers to exploit continual learning for adversarial detection within a knowledge distillation framework. IAC-CAD constructs a sequence of continuous detection tasks which require only a limited number of samples per task. Moreover, the proposed Incremental Attack Configuration (IAC) mechanism selects the representative attacks which maximally cover the entire adversarial feature space and optimizes their training sequence through the Memory-aware Attack Ordering. This design simultaneously mitigates catastrophic forgetting of known attacks and enhances generalization ability against unseen attacks. Extensive experiments verify the superiority and practicality of IAC-CAD.