Fast Dynamic 3D Object Generation from a Single-view Video
Authors:
Preprint
Abstract
Generating dynamic three-dimensional (3D) object from a single-view video is challenging due to the lack of 4D labeled data. Existing methods extend text-to-3D pipelines by transferring off-the-shelf image generation models such as score distillation sampling, but they are slow and expensive to scale (e.g., 150 minutes per object) due to the need for back-propagating the information-limited supervision signals through a large pretrained model. To address this limitation, we propose an efficient video-to-4D object generation framework called Efficient4D. It generates high-quality spacetime-consistent images under different camera views, and then uses them as labeled data to directly train a novel 4D Gaussian splatting model with explicit point cloud geometry, enabling real-time rendering under continuous camera trajectories. Extensive experiments on synthetic and real videos show that Efficient4D offers a remarkable 10-fold increase in speed when compared to prior art alternatives while preserving the same level of innovative view synthesis quality. For example, Efficient4D takes only 14 minutes to model a dynamic object.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- TextMesh: Generation of Realistic 3D Meshes From Text Prompts2024
- Magic3D: High-Resolution Text-to-3D Content Creation2023
- Tensor4D: Efficient Neural 4D Decomposition for High-Fidelity Dynamic Reconstruction and Rendering2023
- Objaverse: A Universe of Annotated 3D Objects2023
- HexPlane: A Fast Representation for Dynamic Scenes2023
- K-Planes: Explicit Radiance Fields in Space, Time, and Appearance2023
- DynIBaR: Neural Dynamic Image-Based Rendering2023
- Real-Time Intermediate Flow Estimation for Video Frame Interpolation2022
- Neural 3D Video Synthesis from Multi-view Video2022
- D-NeRF: Neural Radiance Fields for Dynamic Scenes2021