Open Access
- 19 December 2023
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 7 (4) , 1-24
Abstract
Human mesh reconstruction is essential for various applications, including virtual reality, motion capture, sports performance analysis, and healthcare monitoring. In healthcare contexts such as nursing homes, it is crucial to employ plausible and non-invasive methods for human mesh reconstruction that preserve privacy and dignity. Traditional vision-based techniques encounter challenges related to occlusion, viewpoint limitations, lighting conditions, and privacy concerns. In this research, we present CAvatar, a real-time human mesh reconstruction approach that innovatively utilizes pressure maps recorded by a tactile carpet as input. This advanced, non-intrusive technology obviates the need for cameras during usage, thereby safeguarding privacy. Our approach addresses several challenges, such as the limited spatial resolution of tactile sensors, extracting meaningful information from noisy pressure maps, and accommodating user variations and multiple users. We have developed an attention-based deep learning network, complemented by a discriminator network, to predict 3D human pose and shape from 2D pressure maps with notable accuracy. Our model demonstrates promising results, with a mean per joint position error (MPJPE) of 5.89 cm and a per vertex error (PVE) of 6.88 cm. To the best of our knowledge, we are the first to generate 3D mesh of human activities solely using tactile carpet signals, offering a novel approach that addresses privacy concerns and surpasses the limitations of existing vision-based and wearable solutions. The demonstration of CAvatar is shown at https://youtu.be/ZpO3LEsgV7Y.Keywords
This publication has 64 references indexed in Scilit:
- Smart-surface: Large scale textile pressure sensors arrays for activity recognitionPervasive and Mobile Computing, 2016
- Flexible Piezoelectric Tactile Sensor Array for Dynamic Three-Axis Force MeasurementSensors, 2016
- Flowing ConvNets for Human Pose Estimation in Videos2015
- SMPLACM Transactions on Graphics, 2015
- In-bed posture classification and limb identification2014
- Gait and jump analysis in healthy cats using a pressure mat systemJournal of Feline Medicine and Surgery, 2014
- Capacitive near-field communication for ubiquitous interaction and perception2014
- DeepPose: Human Pose Estimation via Deep Neural Networks2014
- Gait recognition: highly unique dynamic plantar pressure patterns among 104 individualsJournal of The Royal Society Interface, 2011
- Human activity recognition in pervasive health-care: Supporting efficient remote collaborationJournal of Network and Computer Applications, 2008