High-Fidelity 4D Cloth Capture Pipeline with a Two-Level Pattern
Open Access
- 3 July 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 45 (4) , 1-12
Abstract
Capturing cloth motion with high fidelity is challenging due to fine-scale wrinkles, large deformation, and frequent self-occlusion. We present a 4D (spatio-temporal) cloth capture system that achieves 1 mm spatial resolution using only 16 RGB cameras. Our approach uses a two-level marker pattern: sparse, colored L-shaped markers provide robust detection and orientation, while dense noise patterns within each marker enable both marker identification and precise keypoint localization. By unwarping detected markers to a canonical frame, we factor out perspective distortion and most of cloth deformation, allowing the localizer to achieve sub-pixel accuracy. The localized keypoints are triangulated across views to form an incomplete point cloud. A physics-based optimization then deforms a template mesh to match the captured geometry while maintaining penetration-free constraints and physical plausibility for occluded regions. Our method produces temporally coherent sequences that faithfully capture fine wrinkles and folds even during complex motions with self-contact.Keywords
Funding Information
- NSF (2301040)
This publication has 38 references indexed in Scilit:
- DPFlow: Adaptive Optical Flow Estimation with a Dual-Pyramid Framework2025
- GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians2024
- Deep Learning in Virtual Try-On: A Comprehensive SurveyIEEE Access, 2024
- 3D Clothed Human Reconstruction in the Wild2022
- Defining digital fashion: Reshaping the field via a systematic reviewComputers in Human Behavior, 2022
- PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization2020
- TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style2020
- PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization2019
- You Only Look Once: Unified, Real-Time Object Detection2016
- Garment-based motion capture (GaMoCap): high-density capture of human shape in motionMachine Vision and Applications, 2015