EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Vision
Authors:
- Yiming ZhaoYZYiming Zhao
- Taein KwonTKTaein Kwon
- Paul StreliPSPaul Streli
- Marc PollefeysMPMarc Pollefeys
- Christian HolzCHChristian Holz
Preprint
Abstract
Estimating touch contact and pressure in egocentric vision is a central task for downstream applications in Augmented Reality, Virtual Reality, as well as many robotic applications, because it provides precise physical insights into hand-object interaction and object manipulation. However, existing contact pressure datasets lack egocentric views and hand poses, which are essential for accurate estimation during in-situ operation, both for AR/VR interaction and robotic manipulation. In this paper, we introduce EgoPressure,a novel dataset of touch contact and pressure interaction from an egocentric perspective, complemented with hand pose meshes and fine-grained pressure intensities for each contact. The hand poses in our dataset are optimized using our proposed multi-view sequence-based method that processes footage from our capture rig of 8 accurately calibrated RGBD cameras. EgoPressure comprises 5.0 hours of touch contact and pressure interaction from 21 participants captured by a moving egocentric camera and 7 stationary Kinect cameras, which provided RGB images and depth maps at 30 Hz. In addition, we provide baselines for estimating pressure with different modalities, which will enable future developments and benchmarking on the dataset. Overall, we demonstrate that pressure and hand poses are complementary, which supports our intention to better facilitate the physical understanding of hand-object interactions in AR/VR and robotics research.Keywords
All Related Versions
This publication has 28 references indexed in Scilit:
- Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person PerspectivesInternational Journal of Computer Vision, 2025
- ContactArt: Learning 3D Interaction Priors for Category-level Articulated Object and Hand Poses Estimation2024
- Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100International Journal of Computer Vision, 2021
- Learning human–environment interactions using conformal tactile textilesNature Electronics, 2021
- Estimating Fingertip Forces, Torques, and Local Curvatures from Fingernail ImagesRobotica, 2019
- Learning Joint Reconstruction of Hands and Manipulated Objects2019
- Learning the signatures of the human grasp using a scalable tactile gloveNature, 2019
- Capturing Hands in Action Using Discriminative Salient Points and Physics SimulationInternational Journal of Computer Vision, 2016
- Flexible and stretchable fabric-based tactile sensorRobotics and Autonomous Systems, 2015
- Capacitive tactile sensor array for touch screen applicationSensors and Actuators A: Physical, 2011