Hengrui Lou
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Hengrui Lou has a total of 15 co-authors. Here are the top co-authors by number of shared publications:
- Erteng Liu
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China
2 shared publications - Yijun Bei
Zhejiang University, Ningbo, Zhejiang, China
2 shared publications - Zunlei Feng
Zhejiang University, Hangzhou, China
2 shared publications - Hongchang Zhang
School of Software Technology Zhejiang University, Ningbo 310048, P. R. China
1 shared publication - Jie Lei
Zhejiang University of Technology
1 shared publication - Jie Song
Zhejiang University
1 shared publication
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 14712-14721
CGPO: Counterfactual Grounding Policy Optimization for Evidence-Sensitive Pathology Vision-Language Reasoning
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 14712-14721
Open AccessCGPO: Counterfactual Grounding Policy Optimization for Evidence-Sensitive Pathology Vision-Language Reasoning
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 12592-12599
A Large-scale Universal Evaluation Benchmark For Face Forgery Detection
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 12592-12599
Open AccessA Large-scale Universal Evaluation Benchmark For Face Forgery Detection
- 27 May 2025
- Published by World Scientific Pub Co Pte Ltd in International Journal of Pattern Recognition and Artificial Intelligence
- Vol. 39 (09)
One-Step Multi-Frame Inpainting Framework for Real-Time Lip-Sync Digital Human Generation
In recent times, audio-driven lip-synching generation for digital humans has attracted considerable attention. However, the prevailing methodologies frequently encounter challenges pertaining to elevated computational complexity and deficient real-time performance. Although the MuseTalk framework has achieved notable progress in inference efficiency through its end-to-end, latent-space-based single-step generation algorithm, it still suffers from noticeable lip jitter and insufficient synchronization between audio and lip movements. To address these limitations, we propose an enhanced multi-frame inpainting framework that integrates Variational Autoencoders (VAE) and a multi-scale U-Net architecture. Specifically, our approach directly synthesizes the occluded lip region by leveraging multi-frame visual references combined with corresponding audio embeddings, thereby effectively improving lip synchronization and maintaining identity consistency. Furthermore, we introduce a landmark-guided multi-frame sampling strategy designed to enhance model attention towards lip dynamics. To facilitate deeper feature extraction and fusion, we propose a hierarchical latent-space feature fusion network (FusionNet), incorporating global and local residual connections and an enhanced Convolutional Block Attention Module. Additionally, a frame interpolation technique is employed during inference to further smooth lip movements and significantly mitigate lip jitter. The model has been trained on a large-scale Chinese dataset and comprehensively evaluated using both Chinese and English datasets. The experimental results demonstrate that the proposed framework achieves high visual accuracy, consistent lip synchronization, and efficient real-time inference, highlighting its strong cross-lingual generalization capability.
- 27 May 2025
- Published by World Scientific Pub Co Pte Ltd in International Journal of Pattern Recognition and Artificial Intelligence
- Vol. 39 (09)
One-Step Multi-Frame Inpainting Framework for Real-Time Lip-Sync Digital Human Generation
In recent times, audio-driven lip-synching generation for digital humans has attracted considerable attention. However, the prevailing methodologies frequently encounter challenges pertaining to elevated computational complexity and deficient real-time performance. Although the MuseTalk framework has achieved notable progress in inference efficiency through its end-to-end, latent-space-based single-step generation algorithm, it still suffers from noticeable lip jitter and insufficient synchronization between audio and lip movements. To address these limitations, we propose an enhanced multi-frame inpainting framework that integrates Variational Autoencoders (VAE) and a multi-scale U-Net architecture. Specifically, our approach directly synthesizes the occluded lip region by leveraging multi-frame visual references combined with corresponding audio embeddings, thereby effectively improving lip synchronization and maintaining identity consistency. Furthermore, we introduce a landmark-guided multi-frame sampling strategy designed to enhance model attention towards lip dynamics. To facilitate deeper feature extraction and fusion, we propose a hierarchical latent-space feature fusion network (FusionNet), incorporating global and local residual connections and an enhanced Convolutional Block Attention Module. Additionally, a frame interpolation technique is employed during inference to further smooth lip movements and significantly mitigate lip jitter. The model has been trained on a large-scale Chinese dataset and comprehensively evaluated using both Chinese and English datasets. The experimental results demonstrate that the proposed framework achieves high visual accuracy, consistent lip synchronization, and efficient real-time inference, highlighting its strong cross-lingual generalization capability.