HiFi-123: Towards High-fidelity One Image to 3D Content Generation
Authors:
- Yan-Pei CaoYCYan-Pei Cao
Preprint
Abstract
Recent advances in diffusion models have enabled 3D generation from a single image. However, current methods often produce suboptimal results for novel views, with blurred textures and deviations from the reference image, limiting their practical applications. In this paper, we introduce HiFi-123, a method designed for high-fidelity and multi-view consistent 3D generation. Our contributions are twofold: First, we propose a Reference-Guided Novel View Enhancement (RGNV) technique that significantly improves the fidelity of diffusion-based zero-shot novel view synthesis methods. Second, capitalizing on the RGNV, we present a novel Reference-Guided State Distillation (RGSD) loss. When incorporated into the optimization-based image-to-3D pipeline, our method significantly improves 3D generation quality, achieving state-of-the-art performance. Comprehensive evaluations demonstrate the effectiveness of our approach over existing methods, both qualitatively and quantitatively. Video results are available on the project page.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation2023
- Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation2023
- Zero-1-to-3: Zero-shot One Image to 3D Object2023
- Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior2023
- Instant neural graphics primitives with a multiresolution hash encodingACM Transactions on Graphics, 2022
- Vision Transformers for Dense Prediction2021
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis2020
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004