Proxemics-net++: classification of human interactions in still images
Open Access
- 29 April 2024
- journal article
- research article
- Published by Springer Nature in Pattern Analysis and Applications
- Vol. 27 (2) , 1-17
Abstract
Human interaction recognition (HIR) is a significant challenge in computer vision that focuses on identifying human interactions in images and videos. HIR presents a great complexity due to factors such as pose diversity, varying scene conditions, or the presence of multiple individuals. Recent research has explored different approaches to address it, with an increasing emphasis on human pose estimation. In this work, we propose Proxemics-Net++, an extension of the Proxemics-Net model, capable of addressing the problem of recognizing human interactions in images through two different tasks: the identification of the types of “touch codes” or proxemics and the identification of the type of social relationship between pairs. To achieve this, we use RGB and body pose information together with the state-of-the-art deep learning architecture, ConvNeXt, as the backbone. We performed an ablative analysis to understand how the combination of RGB and body pose information affects these two tasks. Experimental results show that body pose information contributes significantly to proxemic recognition (first task) as it allows to improve the existing state of the art, while its contribution in the classification of social relations (second task) is limited due to the ambiguity of labelling in this problem, resulting in RGB information being more influential in this task.Keywords
Funding Information
- Ministerio de Ciencia y Tecnología (PID2019-103871GB-I00, PID2019-103871GB-I00, PID2019-103871GB-I00, PID2019-103871GB-I00)
- Universidad de Córdoba
This publication has 27 references indexed in Scilit:
- Unsupervised open-world human action recognitionPattern Analysis and Applications, 2023
- Proxemics-Net: Automatic Proxemics Recognition in Images2023
- FunnyNet: Audiovisual Learning of Funny Moments in Videos2023
- Human action recognition using a convolutional neural network based on skeleton heatmaps from two-stage pose estimationBiomimetic Intelligence and Robotics, 2022
- HF-SRGR: a new hybrid feature-driven social relation graph reasoning modelThe Visual Computer, 2021
- Deep transfer learning in human–robot interaction for cognitive and physical rehabilitation purposesPattern Analysis and Applications, 2021
- Graph-Based Social Relation Reasoning2020
- Visual Social Relationship RecognitionInternational Journal of Computer Vision, 2020
- Human action recognition in videos with articulated pose information by deep networksPattern Analysis and Applications, 2018
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015