Modeling Ambient Scene Dynamics for Free-view Synthesis
Open Access
- 13 July 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-11
Abstract
We introduce a novel method for dynamic free-view synthesis of an ambient scenes from a monocular capture bringing a immersive quality to the viewing experience. Our method builds upon the recent advancements in 3D Gaussian Splatting (3DGS) that can faithfully reconstruct complex static scenes. Previous attempts to extend 3DGS to represent dynamics have been confined to bounded scenes or require multi-camera captures, and often fail to generalize to unseen motions, limiting their practical application. Our approach overcomes these constraints by leveraging the periodicity of ambient motions to learn the motion trajectory model, coupled with careful regularization. We also propose important practical strategies to improve the visual quality of the baseline 3DGS static reconstructions and to improve memory efficiency critical for GPU-memory intensive learning. We demonstrate high-quality photorealistic novel view synthesis of several ambient natural scenes with intricate textures and fine structural elements. We show that our method significantly outperforms prior methods both qualitatively and quantitatively. Project page: https://ambientgaussian.github.io/Keywords
All Related Versions
This publication has 56 references indexed in Scilit:
- Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis2024
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis2020
- 4D Visualization of Dynamic Events From Unconstrained Multi-View Videos2020
- Pushing the Boundaries of View Extrapolation With Multiplane Images2019
- DeepView: View Synthesis With Learned Gradient Descent2019
- 4D Human Body Correspondences from Panoramic Depth Maps2018
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018
- Robust 3D Human Motion Reconstruction via Dynamic Template Construction2017
- Structure-from-Motion Revisited2016
- A Bayesian Approach for Selective Image-Based Rendering Using Superpixels2015