Spatiotemporal Trident Networks: Detection and Localization of Object Removal Tampering in Video Passive Forensics
- 21 December 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 31 (10) , 4131-4144
Abstract
With the development of video and image processing technology, the field of video tampering forensics is facing enormous challenges. Specifically, as the fundamental basis of judicial forensics, passive forensics for object removal video forgery is particularly essential. To extract tampering traces in video more sufficiently, the author proposed a spatiotemporal trident network based on the spatial rich model (SRM) and 3D convolution (C3D), which provides three branches and can theoretically improve the detection and localization accuracy of tampered regions. Based on the spatiotemporal trident network, a temporal detector and a spatial locator were designed to detect and locate the tampered regions in the temporal and spatial domains of videos. For the temporal detector, 3D CNNs were employed in three branches as the encoders and a bidirectional long short-term memory (BiLSTM) as the decoder. For the spatial locator, a backbone network named C3D-ResNet12 was designed as the encoder of the three branches, and the region proposal networks (RPNs) were employed as the decoders in three branches. In addition, we optimized the loss functions of the above two algorithms based on focal loss and GIoU loss. The experimental results revealed the effectiveness of spatiotemporal detection and localization algorithms: for temporal forgery detection, the accuracy of the frame classification increased to 99+%; for spatial forgery localization, the successful localization rate of the tampered regions in forged frames reached 96+%, and the mean intersection over union of the located tampered regions and the real tampered regions reached 62+%.Keywords
Funding Information
- National Natural Science Foundation of China (62071267)
- Key Research and Development Project of Zhejiang Province China (2020C01165)
This publication has 31 references indexed in Scilit:
- Expecting the Unexpected: Training Detectors for Unusual Pedestrians with Adversarial Imposters2017
- A novel video forgery detection algorithm for blue screen compositing based on 3-stage foreground analysis and trackingMultimedia Tools and Applications, 2017
- Motion-Adaptive Frame Deletion Detection for Digital Video ForensicsIEEE Transactions on Circuits and Systems for Video Technology, 2016
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Online Detection and Classification of Dynamic Hand Gestures with Recurrent 3D Convolutional Neural Networks2016
- GOP based automatic detection of object-based forgery in advanced video2015
- Learning Spatiotemporal Features with 3D Convolutional Networks2015
- Automatic Detection of Object-Based Forgery in Advanced VideoIEEE Transactions on Circuits and Systems for Video Technology, 2015
- Detection of object-based manipulation by the statistical features of object contourForensic Science International, 2014
- Local tampering detection in video sequences2013