HiFi-123: Towards High-fidelity One Image to 3D Content Generation
Authors:
Preprint
Abstract
Recent advances in text-to-image diffusion models have enabled 3D generation from a single image. However, current image-to-3D methods often produce suboptimal results for novel views, with blurred textures and deviations from the reference image, limiting their practical applications. In this paper, we introduce HiFi-123, a method designed for high-fidelity and multi-view consistent 3D generation. Our contributions are twofold: First, we propose a reference-guided novel view enhancement technique that substantially reduces the quality gap between synthesized and reference views. Second, capitalizing on the novel view enhancement, we present a novel reference-guided state distillation loss. When incorporated into the optimization-based image-to-3D pipeline, our method significantly improves 3D generation quality, achieving state-of-the-art performance. Comprehensive evaluations demonstrate the effectiveness of our approach over existing methods, both qualitatively and quantitatively.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- EfficientDreamer: High-Fidelity and Stable 3D Creation via Orthogonal-view Diffusion Priors2024
- HD-Fusion: Detailed Text-to-3D Generation Leveraging Multiple Noise Estimation2024
- Adding Conditional Control to Text-to-Image Diffusion Models2023
- MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing2023
- FateZero: Fusing Attentions for Zero-shot Text-based Video Editing2023
- Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation2023
- Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation2023
- Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators2023
- Zero-1-to-3: Zero-shot One Image to 3D Object2023
- Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior2023