Generative Dataset Distillation Based on Diffusion Model
Preprint
Abstract
This paper presents our method for the generative track of The First Dataset Distillation Challenge at ECCV 2024. Since the diffusion model has become the mainstay of generative models because of its high-quality generative effects, we focus on distillation methods based on the diffusion model. Considering that the track can only generate a fixed number of images in 10 minutes using a generative model for CIFAR-100 and Tiny-ImageNet datasets, we need to use a generative model that can generate images at high speed. In this study, we proposed a novel generative dataset distillation method based on Stable Diffusion. Specifically, we use the SDXL-Turbo model which can generate images at high speed and quality. Compared to other diffusion models that can only generate images per class (IPC) = 1, our method can achieve an IPC = 10 for Tiny-ImageNet and an IPC = 20 for CIFAR-100, respectively. Additionally, to generate high-quality distilled datasets for CIFAR-100 and Tiny-ImageNet, we use the class information as text prompts and post data augmentation for the SDXL-Turbo model. Experimental results show the effectiveness of the proposed method, and we achieved third place in the generative track of the ECCV 2024 DD Challenge. Codes are available at https://github.com/Guang000/BANKO.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- DiM: Distilling Dataset into Generative Model2025
- DDPM-CD: Denoising Diffusion Probabilistic Models as Feature Extractors for Remote Sensing Change Detection2025
- Importance-aware adaptive dataset distillationNeural Networks, 2024
- Diffusion Models: A Comprehensive Survey of Methods and ApplicationsACM Computing Surveys, 2023
- Diffusion Probabilistic Modeling for Video GenerationEntropy, 2023
- A Comprehensive Survey of Dataset DistillationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- Compressed gastric image generation based on soft-label dataset distillation for medical data sharingComputer Methods and Programs in Biomedicine, 2022
- Image Super-Resolution Via Iterative RefinementIEEE Transactions on Pattern Analysis and Machine Intelligence, 2022
- Neural tangent kernel: convergence and generalization in neural networks (invited paper)2021