Toward Adaptive Unsupervised and Blind Image Forgery Localization with ViT-VAE and a Gaussian Mixture Model
Open Access
- 16 July 2025
- journal article
- research article
- Published by MDPI AG in Mathematics
- Vol. 13 (14) , 2285
Abstract
Most image forgery localization methods rely on supervised learning, requiring large labeled datasets for training. Recently, several unsupervised approaches based on the variational autoencoder (VAE) framework have been proposed for forged pixel detection. In these approaches, the latent space is built by a simple Gaussian distribution or a Gaussian Mixture Model. Despite their success, there are still some limitations: (1) A simple Gaussian distribution assumption in the latent space constrains performance due to the diverse distribution of forged images. (2) Gaussian Mixture Models (GMMs) introduce non-convex log-sum-exp functions in the Kullback–Leibler (KL) divergence term, leading to gradient instability and convergence issues during training. (3) Estimating GMM mixing coefficients typically involves either the expectation-maximization (EM) algorithm before VAE training or a multilayer perceptron (MLP), both of which increase computational complexity. To address these limitations, we propose the Deep ViT-VAE-GMM (DVVG) framework. First, we employ Jensen’s inequality to simplify the KL divergence computation, reducing gradient instability and improving training stability. Second, we introduce convolutional neural networks (CNNs) to adaptively estimate the mixing coefficients, enabling an end-to-end architecture while significantly lowering computational costs. Experimental results on benchmark datasets demonstrate that DVVG not only enhances VAE performance but also improves efficiency in modeling complex latent distributions. Our method effectively balances performance and computational feasibility, making it a practical solution for real-world image forgery localization.Keywords
This publication has 38 references indexed in Scilit:
- Image splicing forgery detection: A reviewMultimedia Tools and Applications, 2024
- Semantic-agnostic progressive subtractive network for image manipulation detection and localizationNeurocomputing, 2023
- Toward Deep-Learning-Based Methods in Image Forgery Detection: A SurveyIEEE Access, 2023
- Image manipulation detection by multiple tampering traces and edge artifact enhancementPattern Recognition, 2022
- DVAEGMM: Dual Variational Autoencoder With Gaussian Mixture Model for Anomaly Detection on Attributed NetworksIEEE Access, 2022
- A comprehensive survey on image authentication for tamper detection with localizationMultimedia Tools and Applications, 2022
- Real‐time traffic anomaly detection based on Gaussian mixture model and hidden Markov modelConcurrency and Computation: Practice and Experience, 2021
- CGMVAE: Coupling GMM Prior and GMM Estimator for Unsupervised Clustering and DisentanglementIEEE Access, 2021
- Gaussian Mixture Variational Autoencoder for Semi-Supervised Topic ModelingIEEE Access, 2020
- Using noise inconsistencies for blind image forensicsImage and Vision Computing, 2009