DUAL-Net: Joint Domain-Invariant and User-Adaptive Feature Learning for Gesture Recognition
Open Access
Abstract
Human activity recognition has become an important component of human–computer interaction and ubiquitous computing. Among various sensing technologies, WiFi-based gesture recognition has attracted increasing attention due to its contactless nature and robustness to visual occlusion. However, environmental variations and user-specific differences often lead to significant performance degradation, particularly in cross-user scenarios. Existing methods primarily focus on learning domain-invariant representations, which may overlook user-specific characteristics that are essential for accurate recognition. To address this issue, we propose the Domain-invariant and User-Adaptive Learning Network (DUAL-Net), a dual-branch framework that jointly models domain-invariant and user-adaptive representations. Specifically, DUAL-Net incorporates a contrastive fusion learning (CFL) module with modality-specific encoders to learn complementary representations from WiFi and vision modalities. Furthermore, a spatial matrix difference (SMD)-guided cross-modal generation (CMG) module is introduced to generate user-adaptive WiFi features by incorporating structural priors derived from skeletal representations. To improve deployment efficiency, DUAL-Net adopts a two-stage learning framework, where adaptation is conducted offline to reduce online computational overhead. Experiments on the MM-Fi dataset and a self-collected dataset show that DUAL-Net achieves superior cross-user recognition performance compared with existing single-modality and multimodal methods. In addition, SMD-guided conditioning improves recognition accuracy by up to 8.79% over diffusion generation without structural guidance.Keywords
Funding Information
- National Natural Science Foundation of China (62302466)
- Natural Science Basic Research Program of Shanxi Province (202303021212188)
- Natural Science Basic Research Program of Shanxi Province (202403021222182)
This publication has 53 references indexed in Scilit:
- Review of few-shot learning application in CSI human sensingArtificial Intelligence Review, 2024
- WiFE: WiFi and Vision Based Unobtrusive Emotion Recognition via Gesture and Facial ExpressionIEEE Transactions on Affective Computing, 2023
- Snapture—a Novel Neural Architecture for Combined Static and Dynamic Hand Gesture RecognitionCognitive Computation, 2023
- Multimodal Mutual Attention-Based Sentiment Analysis Framework Adapted to Complicated ContextsIEEE Transactions on Circuits and Systems for Video Technology, 2023
- Everywhere: A Framework for Ubiquitous Indoor LocalizationIEEE Internet of Things Journal, 2022
- Commodity WiFi Sensing in Ten Years: Status, Challenges, and OpportunitiesIEEE Internet of Things Journal, 2022
- Improving human action recognition by jointly exploiting video and WiFi cluesNeurocomputing, 2021
- Multimodal Intelligence: Representation Learning, Information Fusion, and ApplicationsIEEE Journal of Selected Topics in Signal Processing, 2020
- Hand Gesture Recognition in Real Time for Automotive Interfaces: A Multimodal Vision-Based Approach and EvaluationsIEEE Transactions on Intelligent Transportation Systems, 2014
- Smart health: A context-aware health paradigm within smart citiesIEEE Communications Magazine, 2014