Inside-Out Multiperson 3-D Pose Estimation Using the Panoramic Camera Capture System

Authors:
Publisher website
Google Scholar
Abstract
Estimating the 3D human poses of multiple individuals using multiple cameras is a significant research topic within the field of vision-based measurement. Contrary to the classical outside-in camera capture system, the inside-out panoramic camera capture system can cover larger scenes with fewer cameras for 3D human pose estimation. This advancement extends the application of 3D human pose measurement beyond small spaces like motion capture studios. For example, this approach can be utilized for intelligent security surveillance in broad outdoor squares or capturing athletes’ movements in large-scale sports scenes. However, existing inside-out 3D human pose estimation methods that utilize panoramic cameras encounter challenges, particularly in multi-person occlusion scenarios. Aimed at these problems, this paper presents a novel inside-out multi-person 3D human pose estimation method using just a few calibrated panoramic cameras. Specifically, we first propose a cross-view multi-person matching algorithm based on panoramic camera epipolar geometry constraint to improve human body matching robustness across viewpoints. Then, we take advantage of multiple panoramic cameras and introduce a multi-view human pose clustering and fusion algorithm to improve the average recall of 3D human pose estimation. In addition, we propose a multi-view human pose nonlinear optimization algorithm to jointly optimize the weighted reprojection errors of estimated 3D human poses, which can further improve the average precision. We have conducted extensive experiments on the public Panoptic Studio dataset and self-built real and simulated datasets to demonstrate that our method can inside-out estimate 3D human poses using multiple panoramic cameras. Compared to state-of-the-art methods, the 3D human pose omission is greatly reduced through the complementarity of multiple cameras, and the precision of 3D human pose estimation is largely improved by utilizing multi-camera observation information.
Funding Information
  • National Natural Science Foundation of China (62172315, 62073262)
  • Huawei

This publication has 45 references indexed in Scilit: