Information-Containing Adversarial Perturbation for Combating Facial Manipulation Systems
- 29 March 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 18 (15566013) , 2046-2059
Abstract
With the development of deep learning technology, the facial manipulation system has become powerful and easy to use. Such systems can modify the attributes of the given facial images, such as hair color, gender, and age. Malicious applications of such systems pose a serious threat to individuals’ privacy and reputation. Existing studies have proposed various approaches to protect images against facial manipulations. Passive defense methods aim to detect whether the face is real or fake, which works for posterior forensics but can not prevent malicious manipulation. Initiative defense methods protect images upfront by injecting adversarial perturbations into images to disrupt facial manipulation systems but can not identify whether the image is fake. To address the limitation of existing methods, we propose a novel two-tier protection method named Information-containing Adversarial Perturbation (IAP), which provides more comprehensive protection for facial images. We use an encoder to map a facial image and its identity message to a cross-model adversarial example which can disrupt multiple facial manipulation systems to achieve initiative protection. Recovering the message in adversarial examples with a decoder serves passive protection, contributing to provenance tracking and fake image detection. We introduce a feature-level correlation measurement that is more suitable to measure the difference between the facial images than the commonly used mean squared error. Moreover, we propose a spectral diffusion method to spread messages to different frequency channels, thereby improving the robustness of the message against facial manipulation. Extensive experimental results demonstrate that our proposed IAP can recover the messages from the adversarial examples with high average accuracy and effectively disrupt the facial manipulation systems.Keywords
All Related Versions
Funding Information
- National Key Research and Development Program of China (2020AAA0140000)
- Zhejiang Provincial Natural Science Foundation of China (LDT23F01013F01)
- Fundamental Research Funds for the Central Universities
- Alibaba Group through Alibaba Research Intern Program
This publication has 48 references indexed in Scilit:
- HiDDeN: Hiding Data With Deep Networks2018
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018
- Fast Face-Swap Using Convolutional Neural Networks2017
- Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks2017
- Deep Photo Style Transfer2017
- Deep Residual Learning for Image Recognition2016
- Face2Face: Real-Time Face Capture and Reenactment of RGB Videos2016
- Deep Learning Face Attributes in the Wild2015
- Presentation Attack Detection for Face Recognition Using Light Field CameraIEEE Transactions on Image Processing, 2015
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004