Multimodal Mutual Learning for Unsupervised Gait Recognition
- 25 August 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 20 (15566013) , 9207-9219
Abstract
The primary challenge in unsupervised gait recognition lies in generating meaningful and diverse supervisory signals to guide representation learning. The effectiveness of such methods largely depends on the richness of the supervisory signals. Unlike previous methods that construct supervisory signals solely from a single modality, we propose a novel framework, named Multimodal Mutual Learning (M3L), that leverages the identity consistency and complementary nature of both silhouette and skeleton modalities to generate richer and more informative supervisory signals. To fully leverage the richer supervisory signals, M3L encourages mutual prediction between the silhouette and skeleton modalities, guiding the network toward modality-invariant representations. However, mutual prediction alone is hindered by the inherent modality gap, so we introduce a Multimodal Collaborative Module to explicitly bridge this gap and promote cross-modal knowledge transfer. Moreover, to make the framework practical when only one modality is available at inference, we introduce a Multimodal Disentanglement Module. Multimodal Disentanglement Module decouples the two branches and distills a shared representation, preserving the gains of multimodal training while allowing the model to maintain robust performance under single-modality conditions. Extensive experiments on four widely used gait datasets—Gait3D, GREW, CASIA-B, and SUSTech1K—demonstrate the effectiveness of our approach and highlight its potential to advance unsupervised gait recognition.Keywords
Funding Information
- National Natural Science Foundation of China (62206022, 62276025, 62476027)
- Fundamental Research Funds for the Central Universities (2253200026)
This publication has 45 references indexed in Scilit:
- Cut Out the Middleman: Revisiting Pose-Based Gait Recognition2024
- Effective Fusion Method on Silhouette and Pose for Gait RecognitionIEEE Access, 2023
- Learning rich features for gait recognition by integrating skeletons and silhouettesMultimedia Tools and Applications, 2023
- PoseMapGait: A model-based gait recognition method with pose estimation maps and graph convolutional networksNeurocomputing, 2022
- TransGait: Multimodal-based gait recognition with set transformerApplied Intelligence, 2022
- Gait Lateral Network: Learning Discriminative and Compact Representations for Gait Recognition2020
- A model-based gait recognition method with body pose and human prior knowledgePattern Recognition, 2020
- Deep High-Resolution Representation Learning for Human Pose Estimation2019
- A novel density peaks clustering algorithm based on nearest neighbors for improving assignment processPhysica A: Statistical Mechanics and its Applications, 2019
- Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields2017