SS4D: Native 4D Generative Model via Structured Spacetime Latents
Open Access
- 4 December 2025
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 44 (6) , 1-12
Abstract
We present SS4D, a native 4D generative model that synthesizes dynamic 3D objects directly from monocular video. Unlike prior approaches that construct 4D representations by optimizing over 3D or video generative models, we train a generator directly on 4D data, achieving high fidelity, temporal coherence, and structural consistency. At the core of our method is a compressed set of structured spacetime latents. Specifically, (1) To address the scarcity of 4D training data, we build on a pre-trained single-image-to-3D model, preserving strong spatial consistency. (2) Temporal consistency is enforced by introducing dedicated temporal layers that reason across frames. (3) To support efficient training and inference over long video sequences, we compress the latent sequence along the temporal axis using factorized 4D convolutions and temporal downsampling blocks. In addition, we employ a carefully designed training strategy to enhance robustness against occlusion and motion blur, leading to high-quality generation. Extensive experiments show that SS4D produces spatio-temporally consistent 4D objects with superior quality and efficiency, significantly outperforming state-of-the-art methods on both synthetic and real-world datasets.Keywords
All Related Versions
Funding Information
- National Key R&D Program of China (2022ZD0161600)
- Shanghai Artificial Intelligence Laboratory
- Hong Kong RGC TRS (T41-603/20-R)
- the Centre for Perceptual and Interactive Intelligence (CPII) Ltd under the Innovation and Technology Commission (ITC)?s InnoHK
This publication has 15 references indexed in Scilit:
- STAG4D: Spatial-Temporal Anchored Generative 4D Gaussians2024
- Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models2024
- A Unified Approach for Text-and Image-Guided 4D Scene Generation2024
- LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching2024
- RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D2024
- RoFormer: Enhanced transformer with Rotary Position EmbeddingNeurocomputing, 2023
- Objaverse: A Universe of Annotated 3D Objects2023
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows2021
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis2020
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018