Image Sculpting: Precise Object Editing with 3D Geometry Control
Authors:
- 16 June 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4241-4251
Abstract
We present Image Sculpting, a new framework for editing 2D images by incorporating tools from 3D geometry and graphics. This approach differs markedly from existing methods, which are confined to 2D spaces and typically rely on textual instructions, leading to ambiguity and limited control. Image Sculpting converts 2D objects into 3D, enabling direct interaction with their 3D geometry. Post-editing, these objects are re-rendered into 2D, merging into the original image to produce high-fidelity results through a coarse-to-fine enhancement process. The framework supports precise, quantifiable, and physically-plausible editing options such as pose editing, rotation, translation, 3D composition, carving, and serial addition. It marks an initial step towards combining the creative freedom of generative models with the precision of graphics pipelines.Keywords
All Related Versions
This publication has 39 references indexed in Scilit:
- Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation2023
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation2021
- pixelNeRF: Neural Radiance Fields from One or Few Images2021
- Analyzing and Improving the Image Quality of StyleGAN2020
- Interpreting the Latent Space of GANs for Semantic Face Editing2020
- Differentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision2020
- A Style-Based Generator Architecture for Generative Adversarial Networks2019
- Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields2017
- Image-to-Image Translation with Conditional Adversarial Networks2017