Swapped face detection using deep learning and subjective assessment
Authors:
Open Access
- 19 May 2020
- journal article
- research article
- Published by Springer Nature in EURASIP Journal on Information Security
- Vol. 2020 (1) , 1-12
Abstract
The tremendous success of deep learning for imaging applications has resulted in numerous beneficial advances. Unfortunately, this success has also been a catalyst for malicious uses such as photo-realistic face swapping of parties without consent. In this study, we use deep transfer learning for face swapping detection, showing true positive rates greater than 96% with very few false alarms. Distinguished from existing methods that only provide detection accuracy, we also provide uncertainty for each prediction, which is critical for trust in the deployment of such detection systems. Moreover, we provide a comparison to human subjects. To capture human recognition performance, we build a website to collect pairwise comparisons of images from human subjects. Based on these comparisons, we infer a consensus ranking from the image perceived as most real to the image perceived as most fake. Overall, the results show the effectiveness of our method. As part of this study, we create a novel dataset that is, to the best of our knowledge, the largest swapped face dataset created using still images. This dataset will be available for academic research use per request. Our goal of this study is to inspire more research in the field of image forensics through the creation of a dataset and initial analysis.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Automated face swapping and its detection2017
- SwapItUp: A Face Swap Application for Privacy Protection2017
- Deep Residual Learning for Image Recognition2016
- Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer LearningIEEE Transactions on Medical Imaging, 2016
- Deep learning in neural networks: An overviewNeural Networks, 2015
- Video face replacementACM Transactions on Graphics, 2011
- Human Spontaneous Gaze Patterns in Viewing of Faces of Different SpeciesPerception, 2010
- Face swappingACM Transactions on Graphics, 2008
- Exchanging Faces in ImagesComputer Graphics Forum, 2004
- Poisson image editingACM Transactions on Graphics, 2003