Improving Tuning-Free Real Image Editing with Proximal Guidance
Authors:
Preprint
Abstract
DDIM inversion has revealed the remarkable potential of real image editing within diffusion-based methods. However, the accuracy of DDIM reconstruction degrades as larger classifier-free guidance (CFG) scales being used for enhanced editing. Null-text inversion (NTI) optimizes null embeddings to align the reconstruction and inversion trajectories with larger CFG scales, enabling real image editing with cross-attention control. Negative-prompt inversion (NPI) further offers a training-free closed-form solution of NTI. However, it may introduce artifacts and is still constrained by DDIM reconstruction quality. To overcome these limitations, we propose proximal guidance and incorporate it to NPI with cross-attention control. We enhance NPI with a regularization term and reconstruction guidance, which reduces artifacts while capitalizing on its training-free nature. Additionally, we extend the concepts to incorporate mutual self-attention control, enabling geometry and layout alterations in the editing process. Our method provides an efficient and straightforward approach, effectively addressing real image editing tasks with minimal computational overhead.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuning2024
- AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Control2023
- Delta Denoising Score2023
- SVDiff: Compact Parameter Space for Diffusion Fine-Tuning2023
- FateZero: Fusing Attentions for Zero-shot Text-based Video Editing2023
- Editing Implicit Assumptions in Text-to-Image Diffusion Models2023
- Null-text Inversion for Editing Real Images using Guided Diffusion Models2023
- DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation2023
- GLIGEN: Open-Set Grounded Text-to-Image Generation2023
- StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery2021