Monocular 3D Human Pose Estimation for Sports Broadcasts using Partial Sports Field Registration
Preprint
Abstract
The filming of sporting events projects and flattens the movement of athletes in the world onto a 2D broadcast image. The pixel locations of joints in these images can be detected with high validity. Recovering the actual 3D movement of the limbs (kinematics) of the athletes requires lifting these 2D pixel locations back into a third dimension, implying a certain scene geometry. The well-known line markings of sports fields allow for the calibration of the camera and for determining the actual geometry of the scene. Close-up shots of athletes are required to extract detailed kinematics, which in turn obfuscates the pertinent field markers for camera calibration. We suggest partial sports field registration, which determines a set of scene-consistent camera calibrations up to a single degree of freedom. Through joint optimization of 3D pose estimation and camera calibration, we demonstrate the successful extraction of 3D running kinematics on a 400m track. In this work, we combine advances in 2D human pose estimation and camera calibration via partial sports field registration to demonstrate an avenue for collecting valid large-scale kinematic datasets. We generate a synthetic dataset of more than 10k images in Unreal Engine 5 with different viewpoints, running styles, and body types, to show the limitations of existing monocular 3D HPE methods. Synthetic data and code are available at https://github.com/tobibaum/PartialSportsFieldReg_3DHPE.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Extracting spatial knowledge from track and field broadcasts for monocular 3D human pose estimationScientific Reports, 2023
- TVCalib: Camera Calibration for Sports Field Registration in Soccer2023
- Exploiting Temporal Contexts With Strided Transformer for 3D Human Pose EstimationIEEE Transactions on Multimedia, 2022
- Deep reinforcement learning for modeling human locomotion control in neuromechanical simulationJournal of NeuroEngineering and Rehabilitation, 2021
- PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation2021
- MeTRAbs: Metric-Scale Truncation-Robust Heatmaps for Absolute 3D Human Pose EstimationIEEE Transactions on Biometrics, Behavior, and Identity Science, 2020
- Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural EnvironmentsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Sagittal plane kinematics during the transition run in triathletesJournal of Science and Medicine in Sport, 2013
- Flexing Computational Muscle: Modeling and Simulation of Musculotendon DynamicsJournal of Biomechanical Engineering, 2013
- A Kinematics and Kinetic Comparison of Overground and Treadmill RunningMedicine & Science in Sports & Exercise, 2008