Hierarchical Forgery Classifier on Multi-Modality Face Forgery Clues
- 14 August 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 26 (15209210) , 2894-2905
Abstract
Face forgery detection plays an important role in personal privacy and social security. With the development of adversarial generative models, high-quality forgery images become more and more indistinguishable from real to humans. Existing methods always regard as forgery detection task as the common binary or multi-label classification, and ignore exploring diverse multi-modality forgery image types, e.g. visible light spectrum and near-infrared scenarios. In this paper, we propose a novel H ierarchical F orgery C lassifier for M ulti-modality F ace F orgery D etection (HFC-MFFD) , which could effectively learn robust patches-based hybrid domain representation to enhance forgery authentication in multiple modality scenarios. The local hybrid domain representation is designed to explore strong discriminative forgery clues both in the image and frequency domain with the intra-attention mechanism. Furthermore, the specific hierarchical face forgery classifier is designed through the authenticity feedback strategy to integrate diverse discriminative clues. Experimental results on representative multi-modality face forgery datasets demonstrate the superior performance of the proposed HFC-MFFD compared with state-of-the-art algorithms. The source code and models are publicly available at https://github.com/EdWhites/HFC-MFFD.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62276198, U22A2035, U22A2096, 62036007, 61922066, 61906143, 61902297, 61806152, 61876142)
- Key Research and Development Projects of Shaanxi Province (2023-YBGY-231)
- Young Elite Scientists Sponsorship Program (2022QNRC001)
- Guangxi Natural Science Foundation Program (2021GXNSFDA075011)
- Natural Science Basic Research Plan in Shaanxi Province of China (2022JQ-696)
- Technology Innovation Leading Program of Shaanxi (2022QFY01-15)
- CCF-Tencent Open Fund
- Open Research Projects of Zhejiang Lab (2021KG0AB01)
- Fundamental Research Funds for the Central Universities (QTZX23083, XJS221502)
This publication has 68 references indexed in Scilit:
- Automatic Detection of Object-Based Forgery in Advanced VideoIEEE Transactions on Circuits and Systems for Video Technology, 2015
- End-to-End Photo-Sketch Generation via Fully Convolutional Representation Learning2015
- Sparse Representation Based Fisher Discrimination Dictionary Learning for Image ClassificationInternational Journal of Computer Vision, 2014
- The CASIA NIR-VIS 2.0 Face Database2013
- Multi-view Discriminant Analysis2012
- Histograms of Oriented Gradients for Human Detection2005
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Face sketch synthesis and recognition2003
- Multiresolution gray-scale and rotation invariant texture classification with local binary patternsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2002
- Support-vector networksMachine Learning, 1995