GSAlign: Optical Alignment for Photorealistic Dynamic 3D Reconstruction from UGC in Web3

Authors:
Open Access
Publisher website
Google Scholar
Abstract
User-generated 3D content plays a crucial role in enabling truly immersive experiences within the Web3 metaverse. Although significant progress has been made in dynamic 3D content creation and animation using multi-camera setups, they typically require professional equipment and strict synchronization. In contrast, a dynamic and inclusive Web3 ecosystem is collectively built by digital assets from anyone, at any time, and from anywhere. Most Web3 participants are therefore decentralized and non-professional, and their contributed videos, even of the same scene, are usually captured asynchronously using handheld devices. In this paper, we present GSAlign, a novel framework that achieves high-fidelity dynamic 3D reconstruction from unstructured, user-generated videos in the Web3 metaverse. To address the temporal and spatial misalignments inherent in such data, GSAlign integrates three key modules: Epipolar-guided Temporal Alignment (ETA), Motion-centric Separated Reconstruction (MSR), and Background-guided Global Pose Alignment (BGPA). We detail the design of each module and their integration toward a practical end-to-end implementation. Our evaluation of GSAlign on real-world user-generated videos demonstrates robust reconstruction of dynamic 3D scenes despite unsynchronized captures, sparse views, and handheld camera motion.

This publication has 22 references indexed in Scilit: