AIFit: Automatic 3D Human-Interpretable Feedback Models for Fitness Training
Authors:
- 1 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9914-9923
Abstract
I went to the gym today, but how well did I do? And where should I improve? Ah, my back hurts slightly... User engagement can be sustained and injuries avoided by being able to reconstruct 3d human pose and motion, relate it to good training practices, identify errors, and provide early, real-time feedback. In this paper we introduce the first automatic system, AIFit, that performs 3d human sensing for fitness training. The system can be used at home, outdoors, or at the gym. AIFit is able to reconstruct 3d human pose, shape, and motion, reliably segment exercise repetitions, and identify in real-time the deviations between standards learnt from trainers, and the execution of a trainee. As a result, localized, quantitative feedback for correct execution of exercises, reduced risk of injury, and continuous improvement is possible. To support research and evaluation, we introduce the first large scale dataset, Fit3D, containing over 3 million images and corresponding 3d human shape and motion capture ground truth configurations, with over 37 repeated exercises, covering all the major muscle groups, performed by instructors and trainees. Our statistical coach is governed by a global parameter that captures how critical it should be of a trainee’s performance. This is an important aspect that helps adapt to a student’s level of fitness (i.e. beginner vs. advanced vs. expert), or to the expected accuracy of a 3d pose reconstruction method. We show that, for different values of the global parameter, our feedback system based on 3d pose estimates achieves good accuracy compared to the one based on ground-truth motion capture. Our statistical coach offers feedback in natural language, and with spatio-temporal visual grounding.Keywords
This publication has 33 references indexed in Scilit:
- Live-feedback from the IMUs2016
- Never skip leg day: A novel wearable approach to monitoring gym leg exercises2016
- Live Repetition Counting2015
- Virtual Personal Trainer via the Kinect Sensor2015
- High-Tech Tools for Exercise Motivation: Use and Role of Technologies Such as the Internet, Mobile Applications, Social Media, and Video GamesDiabetes Spectrum, 2015
- Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural EnvironmentsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Effects of Exercise and Physical Activity on AnxietyFrontiers in Psychiatry, 2013
- Real-time human pose recognition in parts from single depth imagesCommunications of the ACM, 2013
- Sedentary Behavior: Emerging Evidence for a New Health RiskMayo Clinic Proceedings, 2010
- Robust real-time periodic motion detection, analysis, and applicationsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000