Fast Reconstruction of Monocular Human Video Based on KAN
- 2 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 25 (13) , 24509-24516
Abstract
Creating 3D digital people from monocular video provides many possibilities for a wide range of users and rich applications. In this paper, we propose a fast, high-quality, and effective method for creating 3D digital humans from monocular videos, achieving fast training (2.5 minutes) and real-time rendering. Specifically, we use 3D Gaussian Splatting (3DGS), based on the introduction of Skinned Multi-Person Linear Model (SMPL) human structure prior, and an optimized KAN (Kolmogorov-Arnold-Net) neural network to build effective posture and linear blend skinning (LBS) weight estimation module to quickly and accurately learn the fine details of the 3D human body. In addition, to achieve fast optimization in the densification and prune stages, we propose a two-stage optimization method. First, the local 3D area that needs to be densified is extracted based on LightGlue, and then KL divergence combined with human body prior is further used to guide Gaussian splitting/cloning and merging operations. We conducted extensive experiments on the ZJU MoCap dataset, and the Peak Signal-to-Noise Ratio (PSNR) and Learned Perceptual Image Patch Similarity (LPIPS) metrics indicate that we effectively improved rendering quality while ensuring rendering speed.Keywords
Funding Information
- Hubei Superior and Distinctive Discipline Group of “New Energy Vehicle and Smart Transportation”
This publication has 36 references indexed in Scilit:
- Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis2024
- TeCH: Text-Guided Reconstruction of Lifelike Clothed Humans2024
- Skeleton-based 3D human pose estimation with low-resolution infrared array sensor using attention based CNN-BiGRUInternational Journal of Machine Learning and Cybernetics, 2023
- NeuMan: Neural Human Radiance Field from a Single Video2022
- StyleGAN-Human: A Data-Centric Odyssey of Human Generation2022
- MonoNHR: Monocular Neural Human Renderer2022
- Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans2021
- pixelNeRF: Neural Radiance Fields from One or Few Images2021
- Attention-based context aggregation network for monocular depth estimationInternational Journal of Machine Learning and Cybernetics, 2021
- Structure-from-Motion Revisited2016