Hand-Shadow Poser
Authors:
- Hao XuHXHao Xu
- Yinqiao WangYWYinqiao Wang
- Niloy J. MitraNMNiloy J. Mitra
- Shuaicheng LiuSLShuaicheng Liu
- Pheng-Ann HengPHPheng-Ann Heng
- Chi-Wing FuCFChi-Wing Fu
Preprint
Abstract
Hand shadow art is a captivating art form, creatively using hand shadows to reproduce expressive shapes on the wall. In this work, we study an inverse problem: given a target shape, find the poses of left and right hands that together best produce a shadow resembling the input. This problem is nontrivial, since the design space of 3D hand poses is huge while being restrictive due to anatomical constraints. Also, we need to attend to the input's shape and crucial features, though the input is colorless and textureless. To meet these challenges, we design Hand-Shadow Poser, a three-stage pipeline, to decouple the anatomical constraints (by hand) and semantic constraints (by shadow shape): (i) a generative hand assignment module to explore diverse but reasonable left/right-hand shape hypotheses; (ii) a generalized hand-shadow alignment module to infer coarse hand poses with a similarity-driven strategy for selecting hypotheses; and (iii) a shadow-feature-aware refinement module to optimize the hand poses for physical plausibility and shadow feature preservation. Further, we design our pipeline to be trainable on generic public hand data, thus avoiding the need for any specialized training dataset. For method validation, we build a benchmark of 210 diverse shadow shapes of varying complexity and a comprehensive set of metrics, including a novel DINOv2-based evaluation metric. Through extensive comparisons with multiple baselines and user studies, our approach is demonstrated to effectively generate bimanual hand poses for a large variety of hand shapes for over 85% of the benchmark cases.Keywords
All Related Versions
- Version 1, 2025-05-11, ArXiv
- Published version: ACM Transactions on Graphics, 44 (4), 1.
This publication has 61 references indexed in Scilit:
- Factorized Diffusion: Perceptual Illusions by Noise Decomposition2024
- ARF: Artistic Radiance Fields2022
- InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image2020
- Whole-Body Human Pose Estimation in the Wild2020
- Hand Pose Estimation via Latent 2.5D Heatmap Regression2018
- Capturing Hands in Action Using Discriminative Salient Points and Physics SimulationInternational Journal of Computer Vision, 2016
- Irregular pit placement for dithering images by self-occlusionComputers & Graphics, 2012
- Motion Capture of Hands in Action Using Discriminative Salient Points2012
- Reconstructing 3D Human Pose from 2D Image Landmarks2012
- Generation of view dependent models using free form deformationThe Visual Computer, 2007