From Easy to Hard: Building a Shortcut for Differentially Private Image Synthesis
Authors:
- 12 May 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3988-4006
Abstract
Differentially private (DP) image synthesis aims to generate synthetic images from a sensitive dataset, alleviating the privacy leakage concerns of organizations sharing and utilizing synthetic images. Although previous methods have significantly progressed, especially in training diffusion models on sensitive images with DP Stochastic Gradient Descent (DP-SGD), they still suffer from unsatisfactory performance. In this work, inspired by curriculum learning, we propose a two-stage DP image synthesis framework, where diffusion models learn to generate DP synthetic images from easy to hard. Unlike existing methods that directly use DP-SGD to train diffusion models, we propose an easy stage in the beginning, where diffusion models learn simple features of the sensitive images. To facilitate this easy stage, we propose to use ‘central images’, simply aggregations of random samples of the sensitive dataset. Intuitively, although those central images do not show details, they demonstrate useful characteristics of all images and only incur minimal privacy costs, thus helping early-phase model training. We conduct experiments to present that on the average of four investigated image datasets, the fidelity and utility metrics of our synthetic images are 33.1% and 2.1% better than the state-of-the-art method. The replication package and datasets can be accessed online11.https://github.comJSunnierLee/DP-FETA.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- DAS3D: Dual-Modality Anomaly Synthesis for 3D Anomaly Detection2025
- A Comprehensive Survey of Image Augmentation Techniques for Deep LearningPattern Recognition, 2023
- Randaugment: Practical automated data augmentation with a reduced search space2020
- PPGAN: Privacy-Preserving Generative Adversarial Network2019
- DP-CGAN: Differentially Private Synthetic Data and Label Generation2019
- Rényi Differential Privacy2017
- Deep Learning Face Attributes in the Wild2015
- ImageNet: A large-scale hierarchical image database2009
- Calibrating Noise to Sensitivity in Private Data Analysis2006
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998