Yijun Bei
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Yijun Bei has a total of 32 co-authors. Here are the top co-authors by number of shared publications:
- Zunlei Feng
Zhejiang University, Hangzhou, China
6 shared publications - Erteng Liu
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China
4 shared publications - Kewei Gao3 shared publications
- Hengrui Lou
Zhejiang University, Hangzhou, China
2 shared publications - Lechao Cheng
Hefei University of Technology, Hefei, China
2 shared publications Nianshu Wang
2 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 10904-10912
TraceSleuth: Coarse-to-Fine Hierarchical Reasoning with Trace Amplification for Precise Image Forgery Analysis
The rapid advancement of generative AI has enabled highly realistic image manipulations, posing challenges to trustworthy media analysis. Existing multimodal forgery detection methods often rely on single-stage fusion and one-shot generation, failing to establish reliable cross-modal grounding and leading to inconsistent explanations. We propose TraceSleuth, a coarse-to-fine, evidence-driven framework that explicitly models the formation and verification of forensic evidence. Instead of direct prediction, it first constructs a distribution-aware hypothesis via prototype-based aggregation, and then enforces consistency-aware reasoning through cross-modal alignment for faithful explanation generation. For localization, a graph-based propagation module captures non-local trace consistency and is fused with semantic cues for precise mask prediction. Counterfactual and consistency regularization further ensure that both predictions and explanations are grounded in genuine trace regions. Experiments demonstrate improved detection and localization performance, along with more faithful and interpretable explanations. The code is available at https://github.com/GCLion/TraceSleuth.
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 10904-10912
Open AccessTraceSleuth: Coarse-to-Fine Hierarchical Reasoning with Trace Amplification for Precise Image Forgery Analysis
The rapid advancement of generative AI has enabled highly realistic image manipulations, posing challenges to trustworthy media analysis. Existing multimodal forgery detection methods often rely on single-stage fusion and one-shot generation, failing to establish reliable cross-modal grounding and leading to inconsistent explanations. We propose TraceSleuth, a coarse-to-fine, evidence-driven framework that explicitly models the formation and verification of forensic evidence. Instead of direct prediction, it first constructs a distribution-aware hypothesis via prototype-based aggregation, and then enforces consistency-aware reasoning through cross-modal alignment for faithful explanation generation. For localization, a graph-based propagation module captures non-local trace consistency and is fused with semantic cues for precise mask prediction. Counterfactual and consistency regularization further ensure that both predictions and explanations are grounded in genuine trace regions. Experiments demonstrate improved detection and localization performance, along with more faithful and interpretable explanations. The code is available at https://github.com/GCLion/TraceSleuth.
- Open Access
- 24 February 2026
- Published by Elsevier in SSRN Electronic Journal
PixelEdit: Pixel-Accurate Image Editing with Latent Diffusion Feature Decomposition
The development of diffusion models has led to remarkable performance in synthesizing high-fidelity and diverse images. However, despite their ability to generate high-quality images from varied text prompts, their core mechanism primarily relies on global semantic guidance, which makes it challenging to achieve precise, fine-grained control over the generated content. To address this insufficient controllability, existing methods mainly introduce additional conditional inputs (e.g., image guidance, replaceable text prompts, spatial constraints, and semantic masks) to regulate the generation process. Nevertheless, these methods often treat image content as a coupled whole. They focus primarily on generating images that satisfy given constraints but fail to decouple and independently manipulate internal semantic features. This content coupling limits a user's ability to perform fine-grained, decoupled editing. Targeting the limitations in latent space feature decoupling and fine-grained control, we propose PixelEdit, a diffusion-based framework for pixel-accurate image editing via latent diffusion feature decomposition. PixelEdit's core innovation lies in the unsupervised decomposition and recombination of semantic representations constructed during the diffusion process. Specifically, PixelEdit first extracts features from deep within the diffusion model and clusters them into a set of independent semantic representations. Subsequently, a comparative analysis of features generated with and without text prompt conditioning enables the synthesis of novel composite representations. Finally, by injecting these new features back into the diffusion process, PixelEdit achieves fine-grained image editing. Extensive experiments demonstrate that PixelEdit effectively decomposes images into independently manipulable semantic features, enabling highly refined image editing. Despite being unsupervised, PixelEdit achieves performance comparable or superior to state-of-the-art supervised methods in both controllability and output quality.
- 24 February 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessPixelEdit: Pixel-Accurate Image Editing with Latent Diffusion Feature Decomposition
The development of diffusion models has led to remarkable performance in synthesizing high-fidelity and diverse images. However, despite their ability to generate high-quality images from varied text prompts, their core mechanism primarily relies on global semantic guidance, which makes it challenging to achieve precise, fine-grained control over the generated content. To address this insufficient controllability, existing methods mainly introduce additional conditional inputs (e.g., image guidance, replaceable text prompts, spatial constraints, and semantic masks) to regulate the generation process. Nevertheless, these methods often treat image content as a coupled whole. They focus primarily on generating images that satisfy given constraints but fail to decouple and independently manipulate internal semantic features. This content coupling limits a user's ability to perform fine-grained, decoupled editing. Targeting the limitations in latent space feature decoupling and fine-grained control, we propose PixelEdit, a diffusion-based framework for pixel-accurate image editing via latent diffusion feature decomposition. PixelEdit's core innovation lies in the unsupervised decomposition and recombination of semantic representations constructed during the diffusion process. Specifically, PixelEdit first extracts features from deep within the diffusion model and clusters them into a set of independent semantic representations. Subsequently, a comparative analysis of features generated with and without text prompt conditioning enables the synthesis of novel composite representations. Finally, by injecting these new features back into the diffusion process, PixelEdit achieves fine-grained image editing. Extensive experiments demonstrate that PixelEdit effectively decomposes images into independently manipulable semantic features, enabling highly refined image editing. Despite being unsupervised, PixelEdit achieves performance comparable or superior to state-of-the-art supervised methods in both controllability and output quality.
- Open Access
- 17 February 2026
- Published by Elsevier in SSRN Electronic Journal
Low-Cost Training for Multimodal Large Models via Prompt Tuning and Modal Fusion
Multimodal Large Models have reshaped key industries via multi-sensory collaborative cognition, yet training trillion-parameter MLLMs demands immense computing power and annotated samples, making traditional full-parameter fine-tuning unfit for low-cost scenarios. As a core low-cost approach, prompt tuning suffers from inefficient feature extraction and inadequate modal alignment in multimodal tasks. To solve these, we propose the STMPT-MF method, boosting feature extraction and training efficiency with three key contributions: 1) a progressive feature extraction strategy for efficient feature utilization; 2) a novel cross-modal interaction module and dynamic fusion module that collaboratively realize accurate modal alignment by correcting feature mismatches; 3) extensive systematic experiments that fully validate the method’s effectiveness, superiority and the value of each innovative module.
- 17 February 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessLow-Cost Training for Multimodal Large Models via Prompt Tuning and Modal Fusion
Multimodal Large Models have reshaped key industries via multi-sensory collaborative cognition, yet training trillion-parameter MLLMs demands immense computing power and annotated samples, making traditional full-parameter fine-tuning unfit for low-cost scenarios. As a core low-cost approach, prompt tuning suffers from inefficient feature extraction and inadequate modal alignment in multimodal tasks. To solve these, we propose the STMPT-MF method, boosting feature extraction and training efficiency with three key contributions: 1) a progressive feature extraction strategy for efficient feature utilization; 2) a novel cross-modal interaction module and dynamic fusion module that collaboratively realize accurate modal alignment by correcting feature mismatches; 3) extensive systematic experiments that fully validate the method’s effectiveness, superiority and the value of each innovative module.
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 12592-12599
A Large-scale Universal Evaluation Benchmark For Face Forgery Detection
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 12592-12599
Open AccessA Large-scale Universal Evaluation Benchmark For Face Forgery Detection
- 27 May 2025
- Published by World Scientific Pub Co Pte Ltd in International Journal of Pattern Recognition and Artificial Intelligence
- Vol. 39 (09)
One-Step Multi-Frame Inpainting Framework for Real-Time Lip-Sync Digital Human Generation
In recent times, audio-driven lip-synching generation for digital humans has attracted considerable attention. However, the prevailing methodologies frequently encounter challenges pertaining to elevated computational complexity and deficient real-time performance. Although the MuseTalk framework has achieved notable progress in inference efficiency through its end-to-end, latent-space-based single-step generation algorithm, it still suffers from noticeable lip jitter and insufficient synchronization between audio and lip movements. To address these limitations, we propose an enhanced multi-frame inpainting framework that integrates Variational Autoencoders (VAE) and a multi-scale U-Net architecture. Specifically, our approach directly synthesizes the occluded lip region by leveraging multi-frame visual references combined with corresponding audio embeddings, thereby effectively improving lip synchronization and maintaining identity consistency. Furthermore, we introduce a landmark-guided multi-frame sampling strategy designed to enhance model attention towards lip dynamics. To facilitate deeper feature extraction and fusion, we propose a hierarchical latent-space feature fusion network (FusionNet), incorporating global and local residual connections and an enhanced Convolutional Block Attention Module. Additionally, a frame interpolation technique is employed during inference to further smooth lip movements and significantly mitigate lip jitter. The model has been trained on a large-scale Chinese dataset and comprehensively evaluated using both Chinese and English datasets. The experimental results demonstrate that the proposed framework achieves high visual accuracy, consistent lip synchronization, and efficient real-time inference, highlighting its strong cross-lingual generalization capability.
- 27 May 2025
- Published by World Scientific Pub Co Pte Ltd in International Journal of Pattern Recognition and Artificial Intelligence
- Vol. 39 (09)
One-Step Multi-Frame Inpainting Framework for Real-Time Lip-Sync Digital Human Generation
In recent times, audio-driven lip-synching generation for digital humans has attracted considerable attention. However, the prevailing methodologies frequently encounter challenges pertaining to elevated computational complexity and deficient real-time performance. Although the MuseTalk framework has achieved notable progress in inference efficiency through its end-to-end, latent-space-based single-step generation algorithm, it still suffers from noticeable lip jitter and insufficient synchronization between audio and lip movements. To address these limitations, we propose an enhanced multi-frame inpainting framework that integrates Variational Autoencoders (VAE) and a multi-scale U-Net architecture. Specifically, our approach directly synthesizes the occluded lip region by leveraging multi-frame visual references combined with corresponding audio embeddings, thereby effectively improving lip synchronization and maintaining identity consistency. Furthermore, we introduce a landmark-guided multi-frame sampling strategy designed to enhance model attention towards lip dynamics. To facilitate deeper feature extraction and fusion, we propose a hierarchical latent-space feature fusion network (FusionNet), incorporating global and local residual connections and an enhanced Convolutional Block Attention Module. Additionally, a frame interpolation technique is employed during inference to further smooth lip movements and significantly mitigate lip jitter. The model has been trained on a large-scale Chinese dataset and comprehensively evaluated using both Chinese and English datasets. The experimental results demonstrate that the proposed framework achieves high visual accuracy, consistent lip synchronization, and efficient real-time inference, highlighting its strong cross-lingual generalization capability.
- 1 May 2025
- Published by Elsevier in Pattern Recognition
- Vol. 167
Target-Directed Progressive Gradient Adjusting for transfer learning
- 1 May 2025
- Published by Elsevier in Pattern Recognition
- Vol. 167
Target-Directed Progressive Gradient Adjusting for transfer learning
- 7 March 2025
- Published by Elsevier in Pattern Recognition
- Vol. 164
Behavior capture guided engagement recognition
- 7 March 2025
- Published by Elsevier in Pattern Recognition
- Vol. 164
Behavior capture guided engagement recognition
- Open Access
- 3 December 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-8
Fire and Smoke Detection with Burning Intensity Representation
- 3 December 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-8
Open AccessFire and Smoke Detection with Burning Intensity Representation