Evaluating Step Counting Algorithms on Subsecond Wrist-Worn Accelerometry: A Comparison Using Publicly Available Data Sets
Authors:
- 1 January 2024
- journal article
- research article
- Published by Human Kinetics in Journal for the Measurement of Physical Behaviour
- Vol. 7 (1)
Abstract
Background: Walking-based metrics, including step count and total time walking, are easily interpretable measures of physical activity. Algorithms can estimate steps from accelerometry, which increasingly is measured with accelerometers located on the wrist. However, many existing step counting algorithms have not been validated in free-living settings, exhibit high error rates, or cannot be used without proprietary software. We compare the performance of several existing open-source step counting algorithms on three publicly available data sets, including one with free-living data. Methods: We applied five open-source algorithms: Adaptive Empirical Pattern Transformation, Oak, Step Detection Threshold, Verisense, and stepcount, and one proprietary algorithm (ActiLife) to three publicly available data sets with ground truth step counts: Clemson Ped-Eval, Movement Analysis in Real-World Environments Using Accelerometers, and OxWalk. We evaluate F1 score, precision, recall, mean absolute percent error (MAPE), and mean bias for each algorithm and setting. Results: The machine learning-based stepcount algorithm exhibited the highest F1 score (0.89 ± 0.11) and lowest MAPE (8.6 ± 9%) across all data sets and had the best, or comparable, F1 scores and MAPE in each individual data set. All algorithms performed worse with respect to both F1 score and MAPE in free-living compared with regular walking scenarios, and stepcount and Verisense were most sensitive to sampling frequency of input data. Conclusion: Machine learning-based algorithms, including stepcount, are a promising avenue for step counting. More free-living accelerometry data sets with ground truth step counts are needed for testing, validation, and continued refinement of algorithms.Keywords
This publication has 50 references indexed in Scilit:
- Relationship of Daily Step Counts to All-Cause Mortality and Cardiovascular EventsJACC, 2023
- Prospective Associations of Daily Step Counts and Intensity With Cancer and Cardiovascular Disease Incidence and Mortality and All-Cause MortalityJAMA Internal Medicine, 2022
- Validation of open-source step-counting algorithms for wrist-worn tri-axial accelerometers in cardiovascular patientsGait & Posture, 2022
- The relationships between step count and all-cause mortality and cardiovascular events: A dose–response meta-analysisJournal of Sport and Health Science, 2021
- Association of Daily Step Count and Step Intensity With Mortality Among US AdultsJAMA, 2020
- Association of Step Volume and Intensity With All-Cause Mortality in Older WomenJAMA Internal Medicine, 2019
- Accelerometry Data in Health Research: Challenges and OpportunitiesStatistics in Biosciences, 2019
- Evaluation of the performance of accelerometer-based gait event detection algorithms in different real-world scenarios using the MAREA gait databaseGait & Posture, 2017
- Field evaluation of a random forest activity classifier for wrist-worn accelerometer dataJournal of Science and Medicine in Sport, 2017
- Adherence to the 2008 Adult Physical Activity Guidelines and Mortality RiskAmerican Journal of Preventive Medicine, 2011