Kavli Affiliate: Zheng Zhu
| First 5 Authors: Chaojun Ni, Xiaofeng Wang, Zheng Zhu, Weijie Wang, Haoyun Li
| Summary:
Interactive 3D generation is gaining momentum and capturing extensive
attention for its potential to create immersive virtual experiences. However, a
critical challenge in current 3D generation technologies lies in achieving
real-time interactivity. To address this issue, we introduce WonderTurbo, the
first real-time interactive 3D scene generation framework capable of generating
novel perspectives of 3D scenes within 0.72 seconds. Specifically, WonderTurbo
accelerates both geometric and appearance modeling in 3D scene generation. In
terms of geometry, we propose StepSplat, an innovative method that constructs
efficient 3D geometric representations through dynamic updates, each taking
only 0.26 seconds. Additionally, we design QuickDepth, a lightweight depth
completion module that provides consistent depth input for StepSplat, further
enhancing geometric accuracy. For appearance modeling, we develop FastPaint, a
2-steps diffusion model tailored for instant inpainting, which focuses on
maintaining spatial appearance consistency. Experimental results demonstrate
that WonderTurbo achieves a remarkable 15X speedup compared to baseline
methods, while preserving excellent spatial consistency and delivering
high-quality output.
| Search Query: ArXiv Query: search_query=au:”Zheng Zhu”&id_list=&start=0&max_results=3