DBMNet: Dual-Branch Multi-Modal Network With Spatial-Temporal Fusion for Objective Physical Fatigue Assessment
- 6 October 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Consumer Electronics
- Vol. 71 (4) , 12527-12538
Abstract
Accurate objective physical fatigue (OPF) assessment is critical for enhancing safety in labor-intensive industries, optimizing athletic performance, and supporting personalized health management. With the advancement of wearable sensing technologies, wearable OPF assessment methods based on multi-source information fusion is emerging is gradually becoming an critical approach for sports and health monitoring. However, wearable OPF assessment methods face the following challenges: spatial modeling deficiency, ineffective multimodal fusion and limited long-range dependency capture. To address these limitations, we propose DBMNet, a dual-branch multi-modal network with spatial-temporal fusion for fatigue level classification. DBMNet captures temporal dynamics and spatial patterns separately from raw 12-lead electrocardiogram (ECG) and multi-site inertial measurement unit (IMU) signals. A novel Convolutional Additive Adaptive Cross-refinement Fusion (CAACF) module is introduced to enable adaptive, efficient, and interpretable fusion of heterogeneous modalities. We collected synchronized ECG and IMU data from 65 subjects during treadmill exercise following a modified Bruce protocol, annotated with both coarse- and fine-grained fatigue levels. Extensive experiments demonstrate that DBMNet achieves significant improvements over baseline and state-of-the-art methods, with up to 7% increase in classification accuracy and superior performance across multiple evaluation metrics. Moreover, DBMNet maintains a lightweight architecture suitable for deployment on mobile and wearable devices. This work provides an effective and scalable framework for real-time, objective fatigue monitoring using multi-source physiological signals.Keywords
Funding Information
- Key Research and Development Plan of Guangdong Province (2022B1515120062)
- Shenzhen International Cooperation Project (GJHZ20220913142808016)
- Shenzhen Sustainable Development Special Project (KCXFZ20230731094100001)
- Shenzhen Sustainable Development Special Project (KCXFZ20240903094300001)
- Joint Fund of NSFC and Chongqing (U21A20447)
- Joint Fund of Yeqisun and NSFC (U2241210)
- International Collaboration Fund for Creative Research Teams (W2541011)
This publication has 38 references indexed in Scilit:
- Multi-task joint learning network based on adaptive patch pruning for Alzheimer’s disease diagnosis and clinical score predictionBiomedical Signal Processing and Control, 2024
- Dynamic personalized human body energy expenditure: Prediction using time series forecasting LSTM modelsBiomedical Signal Processing and Control, 2023
- Multimodal integration for data-driven classification of mental fatigue during construction equipment operations: Incorporating electroencephalography, electrodermal activity, and video signalsDevelopments in the Built Environment, 2023
- Identifying mental fatigue of construction workers using EEG and deep learningAutomation in Construction, 2023
- Patch-based deep multi-modal learning framework for Alzheimer’s disease diagnosis using multi-view neuroimagingBiomedical Signal Processing and Control, 2022
- Validity of facial features’ geometric measurements for real-time assessment of mental fatigue in construction equipment operatorsAdvanced Engineering Informatics, 2022
- Worker’s physical fatigue classification using neural networksExpert Systems with Applications, 2022
- Assessment of construction workers’ perceived risk using physiological data from wearable sensors: A machine learning approachJournal of Building Engineering, 2021
- Cardiac fatigue in long-distance runners is associated with ventricular repolarization abnormalitiesHeart Rhythm, 2009
- Maximal oxygen intake and nomographic assessment of functional aerobic impairment in cardiovascular diseaseAmerican Heart Journal, 1973