Weak-Annotation of HAR Datasets using Vision Foundation Models
Open Access
- 5 October 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 55-62
Abstract
As wearable-based data annotation remains, to date, a tedious, time-consuming task requiring researchers to dedicate substantial time, benchmark datasets within the field of Human Activity Recognition in lack richness and size compared to datasets available within related fields. Recently, vision foundation models such as CLIP have gained significant attention, helping the vision community advance in finding robust, generalizable feature representations. With the majority of researchers within the wearable community relying on vision modalities to overcome the limited expressiveness of wearable data and accurately label their to-be-released benchmark datasets offline, we propose a novel, clustering-based annotation pipeline to significantly reduce the amount of data that needs to be annotated by a human annotator. We show that using our approach, the annotation of centroid clips suffices to achieve average labelling accuracies close to 90% across three publicly available HAR benchmark datasets. Using the weakly annotated datasets, we further demonstrate that we can match the accuracy scores of fully-supervised deep learning classifiers across all three benchmark datasets. Code as well as supplementary figures and results are publicly downloadable via github.com/mariusbock/weak_har.Keywords
All Related Versions
Funding Information
- Deutsche Forschungsgemeinschaft (DFG LA 275811-1)
This publication has 33 references indexed in Scilit:
- TriDet: Temporal Action Detection with Relative Boundary Modeling2023
- CLIP4Clip: An empirical study of CLIP for end to end video clip retrieval and captioningNeurocomputing, 2022
- Ego4D: Around the World in 3,000 Hours of Egocentric Video2022
- Video Swin Transformer2022
- Contrastive Self-supervised Learning for Sensor-based Human Activity Recognition2021
- SlowFast Networks for Video Recognition2019
- Unsupervised Learning of Action Classes With Continuous Temporal Embedding2019
- Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset2017
- Collecting complex activity datasets in highly rich networked sensor environments2010
- Activity Recognition from Sparsely Labeled Data Using Multi-Instance Learning2009