Skeleton-Based Two-Person Interaction Recognition Using Multi-Stream Feature Fusion and Joint Evaluator

Authors:

Abstract
Two-person interactions are prevalent in public places, and how to recognize the friendly and violent two-person interactions is crucial for ensuring public safety. However, there are three challenges in two-person interaction recognition: the difficulty in capturing interaction relationships, the interference of interaction-irrelevant actions, and the distortion of the spatial relationship caused by camera view variations. In this paper, we propose an end-to-end multi-stream feature fusion model integrating Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) to capture interaction dynamics and perform fine-grained analysis of interactive actions. The model aims to overcome the limitations of insufficient interaction feature representation through hierarchical fusion of heterogeneous data streams. Meanwhile, to suppress interaction-irrelevant features, we propose a Mutual-Self Activity-level Evaluator (MSEr) to highlight interaction-related features that simultaneously represent individual actions and interaction semantics. Furthermore, to characterize view-invariant features, we employ the 2D Discrete Wavelet Transform (2D-DWT) to analyze action within the spatial-frequency domain. Experimental results demonstrate that the proposed method obtains competitive performance among the compared methods with accuracies of 98.3±0.2%, 94.6±0.1%, and 89.3±0.1% on the SBU-Kinect, NTU RGB+D 60, and NTU RGB+D 120 datasets respectively.
Funding Information
  • National Natural Science Foundation of China (61903170, 62473341)
  • Henan Provincial Science and Technology Research Project (222102210086, 242102220099)
  • Key Scientific Research Projects of Colleges and Universities in Henan Province (25B413005)
  • Natural Science Foundation of Henan Province (252300420389)
  • Henan Province Key Research and Development Project (241111210400)
  • Joint Fund Key Project of Science and Technology Research and Development Plan of Henan Province, China (235200810022)

This publication has 45 references indexed in Scilit: