Trusted Video Inpainting Localization via Deep Attentive Noise Learning
- 5 August 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Dependable and Secure Computing
- Vol. 22 (6) , 7215-7228
Abstract
Digital video inpainting technique has been substantially improved with deep learning in recent years. It may be used as malicious manipulation to remove important objects for creating forged videos. As such it is significant to blindly identify the inpainted regions in videos. In this paper, we present a Trusted Video Inpainting Localization network (TruVIL) with excellent robustness and generalization ability. Observing that high-frequency noise can effectively unveil the inpainted regions, we design deep attentive noise learning in multiple stages to capture the inpainting traces. Firstly, a multiscale noise extraction module based on 3D High Pass (HP3D) layers is used to create the noise modality from input RGB frames. Then the correlation between such two complementary modalities are explored by a cross-modality attentive fusion module to facilitate mutual feature learning. Lastly, spatial details are selectively enhanced by an attentive noise decoding module to boost the localization performance of the network. To prepare enough training samples, we also build a frame-level video object segmentation dataset (VOS2k5) with 2500 videos and pixel-level annotation for all frames. Both quantitative and qualitative evaluations on various inpainted videos verify the robustness against video compression and generalization ability of TruVIL.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62071434, 62262062)
- Fujian Science Fund for Distinguished Young Scholars (2020J06043)
- Beijing Institute of Graphic Communication (Ec202410)
- Fundamental Research Funds for the Central Universities (CUC25GT07)
- CUC Public Computing Cloud
This publication has 43 references indexed in Scilit:
- Deep video inpainting detection and localization based on ConvNeXt dual-stream networkExpert Systems with Applications, 2024
- VIFST: Video Inpainting Localization Using Multi-view Spatial-Frequency Traces2023
- Flow-Guided Transformer for Video Inpainting2022
- XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model2022
- Image manipulation detection by multiple tampering traces and edge artifact enhancementPattern Recognition, 2022
- DVI: Depth Guided Video Inpainting for Autonomous Driving2020
- Learning Joint Spatial-Temporal Transformations for Video Inpainting2020
- Flow-edge Guided Video Completion2020
- YouTube-VOS: Sequence-to-Sequence Video Object Segmentation2018
- Using noise inconsistencies for blind image forensicsImage and Vision Computing, 2009