Effective Real Image Editing with Accelerated Iterative Diffusion Inversion
Authors:
- 1 October 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 15866-15875
Abstract
Despite all recent progress, it is still challenging to edit and manipulate natural images with modern generative models. When using Generative Adversarial Network (GAN), one major hurdle is in the inversion process mapping a real image to its corresponding noise vector in the latent space, since it is necessary to be able to reconstruct an image to edit its contents. Likewise for Denoising Diffusion Implicit Models (DDIM), the linearization assumption in each inversion step makes the whole deterministic inversion process unreliable. Existing approaches that have tackled the problem of inversion stability often incur in significant trade-offs in computational efficiency. In this work we propose an Accelerated Iterative Diffusion Inversion method, dubbed AIDI, that significantly improves reconstruction accuracy with minimal additional overhead in space and time complexity. By using a novel blended guidance technique, we show that effective results can be obtained on a large range of image editing tasks without large classifier-free guidance in inversion. Furthermore, when compared with other diffusion inversion based works, our proposed process is shown to be more robust for fast image editing in the 10 and 20 diffusion steps’ regimes.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- HyperInverter: Improving StyleGAN Inversion via Hypernetwork2022
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation2022
- Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation2021
- Analyzing and Improving the Image Quality of StyleGAN2020
- Image2StyleGAN++: How to Edit the Embedded Images?2020
- Editing in Style: Uncovering the Local Semantics of GANs2020
- StarGAN v2: Diverse Image Synthesis for Multiple Domains2020
- A Style-Based Generator Architecture for Generative Adversarial Networks2019
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018