Text-to-3D using Gaussian Splatting
Authors:
Top Cited Papers
- 16 June 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 21401-21412
Abstract
Automatic text-to-3D generation that combines Score Distillation Sampling (SDS) with the optimization of volume rendering has achieved remarkable progress in synthesizing realistic 3D objects. Yet most existing text-to-3D methods by SDS and volume rendering suffer from inaccurate geometry, e.g., the Janus issue, since it is hard to explicitly integrate 3D priors into implicit 3D representations. Besides, it is usually time-consuming for them to generate elaborate 3D models with rich colors. In response, this paper proposes GSGEN, a novel method that adopts Gaussian Splatting, a recent state-of-the-art representation, to text-to-3D generation. GSGEN aims at generating high-quality 3D objects and addressing existing shortcomings by exploiting the explicit nature of Gaussian Splatting that enables the incorporation of 3D prior. Specifically, our method adopts a progressive optimization strategy, which includes a geometry optimization stage and an appearance refinement stage. In geometry optimization, a coarse representation is established under 3D point cloud diffusion prior along with the ordinary 2D SDS optimization, ensuring a sensible and 3D-consistent rough shape. Subsequently, the obtained Gaussians undergo an iterative appearance refinement to enrich texture details. In this stage, we increase the number of Gaussians by compactness-based densification to enhance continuity and improve fidelity. With these designs, our approach can generate 3D assets with delicate details and accurate geometry. Extensive evaluations demonstrate the effectiveness of our method, especially for capturing high-frequency components. Our code is available at https://github.com/gsgen3d/gsgen.Keywords
All Related Versions
This publication has 37 references indexed in Scilit:
- The general method of the tanker car mouth pose measurementRobotic Intelligence and Automation, 2023
- Low-cost AI-based solar panel detection drone design and implementation for solar power systemsRobotic Intelligence and Automation, 2023
- TensoRF: Tensorial Radiance Fields2022
- Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction2022
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields2022
- Hallucinated Neural Radiance Fields in the Wild2022
- Block-NeRF: Scalable Large Scene Neural View Synthesis2022
- D-NeRF: Neural Radiance Fields for Dynamic Scenes2021
- The farthest point strategy for progressive image samplingIEEE Transactions on Image Processing, 1997