Abstract
3D gait representation, intrinsically tied to identity, is of great importance in tackling the challenge of video-based clothes-changing person re-identification (VCCRe-ID), where individuals’ clothing appearances vary across video clips captured over an extended duration. Most existing methods derive 3D gait features from sequential 3D skeletons or parameters of a predefined 3D human model (e.g. SMPL). However, the sparsity of 3D skeletons and the low-dimensional nature of 3D human model parameters not only limit their expressive power and discriminative ability, but also impede alignment with higher-dimensional appearance features. To address these issues, we propose Structured Distilled 3D Gait Fields (SD-3DGF) to learn better aligned, more expressive, and discriminative 3D gait features for VCCRe-ID. SD-3DGF defines distilled feature fields that evolve over time, allowing the simulation of 3D gait dynamics from temporal 2D descriptors. Specifically, it densely transfers frame-level shape knowledge from structured 3D space (i.e., 3D mesh surface) to the 2D image domain via 3D-2D knowledge distillation, and models local spatiotemporal correlations among the distilled features to extract 3D gait features. This learning paradigm makes 3D gait features more compatible with appearance features, as they are naturally aligned in the spatiotemporal domain. To fully exploit this potential, we develop a novel Modality-Representative Interactive Encoding (MRIE) module, which facilitates efficient gait-appearance interaction and joint encoding via global and local representative tokens. Experiments on both VCCRe-ID and standard video-based Re-ID benchmarks demonstrate the effectiveness of our method. The code is available at https://github.com/yubinwang2021/SD-3DGF.
Funding Information
  • the Research Project of ZJU-League Research and Development Center, Zhejiang Laboratory (2019KD0AB01)