4D-Rotor Gaussian Splatting: Towards Efficient Novel View Synthesis for Dynamic Scenes
Top Cited Papers
Open Access
- 13 July 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-11
Abstract
We consider the problem of novel-view synthesis (NVS) for dynamic scenes. Recent neural approaches have accomplished exceptional NVS results for static 3D scenes, but extensions to 4D time-varying scenes remain non-trivial. Prior efforts often encode dynamics by learning a canonical space plus implicit or explicit deformation fields, which struggle in challenging scenarios like sudden movements or generating high-fidelity renderings. In this paper, we introduce 4D Gaussian Splatting (4DRotorGS), a novel method that represents dynamic scenes with anisotropic 4D XYZT Gaussians, inspired by the success of 3D Gaussian Splatting in static scenes [Kerbl et al. 2023]. We model dynamics at each timestamp by temporally slicing the 4D Gaussians, which naturally compose dynamic 3D Gaussians and can be seamlessly projected into images. As an explicit spatial-temporal representation, 4DRotorGS demonstrates powerful capabilities for modeling complicated dynamics and fine details—especially for scenes with abrupt motions. We further implement our temporal slicing and splatting techniques in a highly optimized CUDA acceleration framework, achieving real-time inference rendering speeds of up to 277 FPS on an RTX 3090 GPU and 583 FPS on an RTX 4090 GPU. Rigorous evaluations on scenes with diverse motions showcase the superior efficiency and effectiveness of 4DRotorGS, which consistently outperforms existing methods both quantitatively and qualitatively.Keywords
All Related Versions
Funding Information
- National Key R&D Program of China (2022ZD0160801)
This publication has 55 references indexed in Scilit:
- TensoRF: Tensorial Radiance Fields2022
- ColMap: A memory-efficient occupancy grid mapping frameworkRobotics and Autonomous Systems, 2021
- Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes2021
- Free View Synthesis2020
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis2020
- Multi-View Neural Human Rendering2020
- Pushing the Boundaries of View Extrapolation With Multiplane Images2019
- DeepView: View Synthesis With Learned Gradient Descent2019
- Let There Be Color! Large-Scale Texturing of 3D Reconstructions2014
- Photorealistic Scene Reconstruction by Voxel ColoringInternational Journal of Computer Vision, 1999