Drag Your Noise: Interactive Point-based Editing via Diffusion Semantic Propagation
Authors:
- 16 June 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6743-6752
Abstract
Point-based interactive editing serves as an essential tool to complement the controllability of existing generative mod-els. A concurrent work, DragD iffus ion, updates the diffusion latent map in response to user inputs, causing global latent map alterations. This results in imprecise preservation of the original content and unsuccessful editing due to gradient vanishing. In contrast, we present DragNoise, offering ro-bust and accelerated editing without retracing the latent map. The core rationale of DragNoise lies in utilizing the predicted noise output of each U-Net as a semantic editor. This approach is grounded in two critical observations: firstly, the bottleneck features of U-Net inherently possess semantically rich features ideal for interactive editing; secondly, high-level semantics, established early in the denoising process, show minimal variation in subsequent stages. Leveraging these insights, DragNoise edits diffusion semantics in a sin-gle denoising step and efficiently propagates these changes, ensuring stability and efficiency in diffusion editing. Compar-ative experiments reveal that DragNoise achieves superior control and semantic retention, reducing the optimization time by over 50% compared to DragDiffusion. Our codes are available at https://github.com/haofenglIDragNoise.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- Editing Out-of-Domain GAN Inversion via Differential Activations2022
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- Blended Diffusion for Text-driven Editing of Natural Images2022
- Discovering Interpretable Latent Space Directions of GANs Beyond Binary Attributes2021
- HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color Histograms2021
- In-Domain GAN Inversion for Real Image Editing2020
- Analyzing and Improving the Image Quality of StyleGAN2020
- A Style-Based Generator Architecture for Generative Adversarial Networks2019
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015