Wonder3D: Single Image to 3D Using Cross-Domain Diffusion
Authors:
Top Cited Papers
- 16 June 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9970-9980
Abstract
In this work, we introduce Wonder3D, a novel method for efficiently generating high-fidelity textured meshes from single-view images. Recent methods based on Score Distillation Sampling (SDS) have shown the potential to recover 3D geometry from 2D diffusion priors, but they typically suffer from time-consuming per-shape optimization and inconsistent geometry. In contrast, certain works di-rectly produce 3D information via fast network inferences, but their results are often of low quality and lack geometric details. To holistically improve the quality, consistency, and efficiency of single-view reconstruction tasks, we pro-pose a cross-domain diffusion model that generates multi-view normal maps and the corresponding color images. To ensure the consistency of generation, we employ a multi-view cross-domain attention mechanism that facilitates information exchange across views and modalities. Lastly, we introduce a geometry-aware normal fusion algorithm that extracts high-quality surfaces from the multi-view 2D representations in only 2 r-;» 3 minutes. Our extensive evaluations demonstrate that our method achieves high-quality reconstruction results, robust generalization, and good efficiency compared to prior works.Keywords
All Related Versions
This publication has 42 references indexed in Scilit:
- Control3Diff: Learning Controllable 3D Diffusion Models from Single-view Images2024
- TextMesh: Generation of Realistic 3D Meshes From Text Prompts2024
- Magic3D: High-Resolution Text-to-3D Content Creation2023
- SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Views2022
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- Zero-Shot Text-Guided Object Generation with Dream Fields2022
- Neural Rays for Occlusion-aware Image-based Rendering2022
- Diffusion Probabilistic Models for 3D Point Cloud Generation2021
- A Survey on Learning-Based Robotic GraspingCurrent Robotics Reports, 2020
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018