ViPS: Video-informed Pose Spaces for Auto-Rigged Meshes
Open Access
- 9 November 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 45 (6) , 1-21
Abstract
Kinematic rigs provide a structured interface for articulating 3D meshes but lack any associated pose space , i.e., an explicit representation of the plausible manifold of joint configurations for a given mesh. Without such a pose space, stochastic sampling or manual manipulation of raw rig parameters easily results in semantic and/or geometric violations, such as anatomical hyperextension and non-physical self-intersections. We propose Video-informed Pose Spaces (ViPS), a feed-forward framework that discovers the latent distribution of valid articulations for (auto-)rigged meshes by distilling motion priors from a pretrained video diffusion model. Unlike existing methods that rely on scarce, artist-authored 4D datasets, or focus on reconstructing instances of individual motions, ViPS transfers generative video model priors into a universal distribution over the given rig parameterization. Differentiable geometric constraints on the skinned mesh naturally preserve shape-specific geometric consistency without requiring additional (hand-coded) mesh regularization. Our feed-forward model reveals a smooth, compact, and controllable pose space. This, in turn, supports sampling for diverse shape variations, manifold projection for inverse kinematics, and temporally coherent trajectories for animation and keyframing. Further, the distilled 3D pose samples serve as semantic proxies to guide video diffusion, effectively closing the loop between generative 2D priors and structured 3D kinematic control. Our evaluations show that ViPS, trained solely using video priors, matches the performance of state-of-the-art models trained on synthetic artist-created 4D data in both plausibility and diversity. Additionally, as a universal model, ViPS exhibits robust zero-shot generalization to out-of-distribution species and unseen skeletal topologies. Project Page: https://honglin-c.github.io/vips/Keywords
This publication has 23 references indexed in Scilit:
- DIMO: Diverse 3D Motion Generation for Arbitrary Objects2025
- PhysRig: Differentiable Physics-Based Skinning and Rigging Framework for Realistic Articulated Object Modeling2025
- Gaussian Variation Field Diffusion for High-Fidelity Video-to-4D Synthesis2025
- 4D-Fly: Fast 4D Reconstruction from a Single Monocular Video2025
- Motion Modes: What Could Happen Next?2025
- Neural Implicit Representation for Building Digital Twins of Unknown Articulated Objects2024
- Motion2VecSets: 4D Latent Vector Set Diffusion for Non-Rigid Shape Reconstruction and Tracking2024
- Skinned Motion Retargeting with Residual Perception of Motion Semantics & Geometry2023
- D-NeRF: Neural Radiance Fields for Dynamic Scenes2021
- A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation2016