Shake to Leak: Fine-tuning Diffusion Models Can Amplify the Generative Privacy Risk
Authors:
- 9 April 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 18-32
Abstract
While diffusion models have recently demonstrated remarkable progress in generating realistic images, privacy risks also arise: published models or APIs could generate training images and thus leak privacy-sensitive training information. In this paper, we reveal a new risk, Shake-to-Leak (S2L), that fine-tuning the pre-trained models with manipulated data can amplify the existing privacy risks. We demonstrate that S2L could occur in various standard fine-tuning strategies for diffusion models, including concept-injection methods (DreamBooth and Textual Inversion) and parameter-efficient methods (LoRA and Hypernetwork), as well as their combinations. In the worst case, S2L can amplify the state-of-the-art membership inference attack (MIA) on diffusion models by 5.4% (absolute difference) AUC and can increase extracted private samples from almost 0 samples to 16.3 samples on average per target domain. This discovery underscores that the privacy risk with diffusion models is even more severe than previously recognized. Codes are available at https://github.com/VITA-Group/Shake-to-Leak.Keywords
All Related Versions
Funding Information
- National Science Foundation
- National Science Foundation
- National Aeronautics and Space Administration
This publication has 15 references indexed in Scilit:
- Loss and Likelihood Based Membership Inference of Diffusion Models2023
- Inversion-based Style Transfer with Diffusion Models2023
- DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation2023
- Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models2023
- Diffusion models in medical imaging: A comprehensive surveyMedical Image Analysis, 2023
- Brain Imaging Generation with Latent Diffusion Models2022
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- Adversarial Mask Generation for Preserving Visual Privacy2021
- Learning privacy-enhancing face representations through feature disentanglement2020
- Calibrating Noise to Sensitivity in Private Data Analysis2006