WEAR: An Outdoor Sports Dataset for Wearable and Egocentric Activity Recognition
Open Access
- 21 November 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 8 (4) , 1-21
Abstract
Research has shown the complementarity of camera- and inertial-based data for modeling human activities, yet datasets with both egocentric video and inertial-based sensor data remain scarce. In this paper, we introduce WEAR, an outdoor sports dataset for both vision- and inertial-based human activity recognition (HAR). Data from 22 participants performing a total of 18 different workout activities was collected with synchronized inertial (acceleration) and camera (egocentric video) data recorded at 11 different outside locations. WEAR provides a challenging prediction scenario in changing outdoor environments using a sensor placement, in line with recent trends in real-world applications. Benchmark results show that through our sensor placement, each modality interestingly offers complementary strengths and weaknesses in their prediction performance. Further, in light of the recent success of single-stage Temporal Action Localization (TAL) models, we demonstrate their versatility of not only being trained using visual data, but also using raw inertial data and being capable to fuse both modalities by means of simple concatenation. The dataset and code to reproduce experiments is publicly available via: mariusbock.github.io/wear/.Keywords
All Related Versions
Funding Information
- German Research Foundation (506589320)
This publication has 72 references indexed in Scilit:
- Proposal-Free Temporal Action Detection via Global Segmentation Mask Learning2022
- ReAct: Temporal Action Detection with Relational Queries2022
- ActionFormer: Localizing Moments of Actions with Transformers2022
- TallFormer: Temporal Action Localization with a Long-Memory Transformer2022
- Towards a multimodal human activity dataset for healthcareMultimedia Systems, 2022
- Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100International Journal of Computer Vision, 2021
- Bottom-Up Temporal Action Localization with Mutual Regularization2020
- Boundary Content Graph Neural Network for Temporal Action Proposal Generation2020
- A Multi-media Exchange Format for Time-Series Dataset Curation2019
- Activity Recognition from User-Annotated Acceleration Data2004