I am a Smartwatch and I can Track my User's Arm
Authors:
Open Access
- 20 June 2016
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 85-96
Abstract
This paper aims to track the 3D posture of the entire arm - both wrist and elbow - using the motion and magnetic sensors on smartwatches. We do not intend to employ machine learning to train the system on a specific set of gestures. Instead, we aim to trace the geometric motion of the arm, which can then be used as a generic platform for gesture-based applications. The problem is challenging because the arm posture is a function of both elbow and shoulder motions, whereas the watch is only a single point of (noisy) measurement from the wrist. Moreover, while other tracking systems (like indoor/outdoor localization) often benefit from maps or landmarks to occasionally reset their estimates, such opportunities are almost absent here. While this appears to be an under-constrained problem, we find that the pointing direction of the forearm is strongly coupled to the arm's posture. If the gyroscope and compass on the watch can be made to estimate this direction, the 3D search space can become smaller; the IMU sensors can then be applied to mitigate the remaining uncertainty. We leverage this observation to design ArmTrak, a system that fuses the IMU sensors and the anatomy of arm joints into a modified hidden Markov model (HMM) to continuously estimate state variables. Using Kinect 2.0 as ground truth, we achieve around 9.2 cm of median error for free-form postures; the errors increase to 13.3 cm for a real time version. We believe this is a step forward in posture tracking, and with some additional work, could become a generic underlay to various practical applications.Keywords
This publication has 33 references indexed in Scilit:
- Real-time human pose recognition in parts from single depth imagesCommunications of the ACM, 2013
- A New Method for Measuring Meal Intake in Humans via Automated Wrist Motion TrackingApplied Psychophysiology and Biofeedback, 2012
- Motion reconstruction using sparse accelerometer dataACM Transactions on Graphics, 2011
- A general method for human activity recognition in videoComputer Vision and Image Understanding, 2006
- A survey of advances in vision-based human motion capture and analysisComputer Vision and Image Understanding, 2006
- Inertial measurements of upper limb motionMedical & Biological Engineering & Computing, 2006
- Pictorial Structures for Object RecognitionInternational Journal of Computer Vision, 2005
- Recent developments in human motion analysisPattern Recognition, 2003
- Towards Model-Based Recognition of Human Movements in Image SequencesCVGIP: Image Understanding, 1994
- Model-based vision: a program to see a walking personImage and Vision Computing, 1983