Dual-Domain Triple Contrast for Cross-Dataset Skeleton-Based Action Recognition
Open Access
- 10 March 2025
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 21 (4) , 1-23
Abstract
Skeleton-based Action Recognition (SAR) is widely recognized for its robustness and efficiency in human action analysis, but its performance in cross-dataset tasks has been limited due to domain shifts between different datasets. To address this challenge, current methods typically approach cross-dataset SAR as an Unsupervised Domain Adaptation (UDA) task, which is tackled using domain adaptation or self-supervised learning strategies. In this article, we propose a Dual-Domain Triple Contrast (D2TC) framework for cross-dataset SAR under the UDA setting. Unlike existing UDA methods that either focus on a single strategy or superficially combine strategies, our D2TC leverages contrastive learning to integrate both strategies into a unified framework. It performs three types of contrastive learning: Self-Supervised Contrastive Learning, Supervised Contrastive Learning, and UDA with Contrastive Learning, across both source and target domains. The triple contrasts go beyond mere summation, effectively bridging the domain gap and enhancing the model’s representational capacity. Additionally, we introduce multi-modal ensemble contrast and extreme skeleton augmentation methods to further enhance the skeleton-based representation learning. Extensive experiments on six cross-dataset settings validate the superiority of our D2TC framework over state-of-the-art methods, demonstrating its effectiveness in reducing domain discrepancies and improving cross-dataset SAR performance. The codes are available on https://github.com/KennCoder7/DualDomainTripleContrast .Keywords
Funding Information
- National Natural Science Foundation of China (62472092, 62172089, 62106045)
- Natural Science Foundation of Jiangsu Province (BK20241751)
- Jiangsu Provincial Key Laboratory of Computer Networking Technology
- Jiangsu Provincial Key Laboratory of Network and Information Security (BM2003201)
- Key Laboratory of Computer Network and Information Integration of Ministry of Education of China (93K-9)
- Nanjing Purple Mountain Laboratories
This publication has 22 references indexed in Scilit:
- EnsCLR: Unsupervised skeleton-based action recognition via ensemble contrastive learning of representationComputer Vision and Image Understanding, 2024
- DeGCN: Deformable Graph Convolutional Networks for Skeleton-Based Action RecognitionIEEE Transactions on Image Processing, 2024
- NaCL: noise-robust cross-domain contrastive learning for unsupervised domain adaptationMachine Learning, 2023
- Human Action Recognition and Prediction: A SurveyInternational Journal of Computer Vision, 2022
- Skeleton-based action recognition via spatial and temporal transformer networksComputer Vision and Image Understanding, 2021
- A Review of Single-Source Deep Unsupervised Visual Domain AdaptationIEEE Transactions on Neural Networks and Learning Systems, 2020
- MS2L2020
- Graph Interaction Networks for Relation Transfer in Human Activity VideosIEEE Transactions on Circuits and Systems for Video Technology, 2020
- View Adaptive Neural Networks for High Performance Skeleton-Based Human Action RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2019
- Enhanced skeleton visualization for view invariant human action recognitionPattern Recognition, 2017