WonderTurbo: Generating Interactive 3D World in 0.72 Seconds

Authors:
  • CNChaojun Ni
    Chaojun Ni
  • XWXiaofeng Wang
    Xiaofeng Wang
  • ZZZheng Zhu
    Zheng Zhu
  • WWWeijie Wang
    Weijie Wang
  • HLHaoyun Li
    Haoyun Li
  • GZGuosheng Zhao
    Guosheng Zhao
  • JLJie Li
    Jie Li
  • WQWenkang Qin
    Wenkang Qin
  • GHGuan Huang
    Guan Huang
  • WMWenjun Mei
    Wenjun Mei

Abstract
Interactive 3D generation is gaining momentum and capturing extensive attention for its potential to create immersive virtual experiences. However, a critical challenge in current 3D generation technologies lies in achieving real-time interactivity. To address this issue, we introduce WonderTurbo, the first real-time interactive 3D scene generation framework capable of generating novel perspectives of 3D scenes within 0.72 seconds. Specifically, WonderTurbo accelerates both geometric and appearance modeling in 3D scene generation. In terms of geometry, we propose StepSplat, an innovative method that constructs efficient 3D geometric representations through dynamic updates, each taking only 0.26 seconds. Additionally, we design QuickDepth, a lightweight depth completion module that provides consistent depth input for StepSplat, further enhancing geometric accuracy. For appearance modeling, we develop FastPaint, a 2-steps diffusion model tailored for instant inpainting, which focuses on maintaining spatial appearance consistency. Experimental results demonstrate that WonderTurbo achieves a remarkable 15X speedup compared to baseline methods, while preserving excellent spatial consistency and delivering high-quality output.
All Related Versions