On Space-Time Filtering Framework for Matching Human Actions Across Different Viewpoints
Authors:
- 23 October 2017
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 27 (3) , 1230-1242
Abstract
Space-time template matching is considered as a promising approach for human action recognition. However, a major drawback of template based methods is computational overhead due to matching in spatial domain. Recently, spacetime correlation based action filters have been proposed for recognizing human actions in frequency domain. These action filters present reduction in time complexity as Fourier transform based matching is faster than spatial template matching. However, the utility of such action filters is challenged due to a number of factors: (i) inability to deal with view variations due to implicit lack of support for viewinvariance; (ii) these filters can be trained only for one action class at a time and separate filters are required for each action class with increased computational overhead; (iii) these filters simply take average of similar action instances and behave no better than average filters; and (iv) slightly misaligned action datasets create problems as these filters are not shiftinvariant. In this paper, we try to address these shortcomings by proposing an advanced space-time filtering framework for recognizing human actions despite large viewpoint variations. Rather than using crude intensity values, we use 3D tensor structure at each pixel which characterizes the most common local motion in action sequences. Discrete Tensor Fourier Transform (DTFT) is then applied to achieve frequency domain representations. We then,form view clusters from multiple view action data and use space-time correlation filtering to achieve discriminative view representations. These representations are used in an innovative way to achieve action recognition despite viewpoint variations. A detailed experimentation on multiple view action datasets including IXMAS, WVU and N-UCLA action dataset indicates that our approach works equally well for RGB and RGB-D video data with increased accuracy and efficiency.Keywords
This publication has 39 references indexed in Scilit:
- Zero-Aliasing Correlation Filters for Object RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Multitask Linear Discriminant Analysis for View Invariant Action RecognitionIEEE Transactions on Image Processing, 2014
- High-Speed Tracking with Kernelized Correlation FiltersIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Dense Trajectories and Motion Boundary Descriptors for Action RecognitionInternational Journal of Computer Vision, 2013
- Matching actions in presence of camera motionComputer Vision and Image Understanding, 2006
- Free viewpoint action recognition using motion history volumesComputer Vision and Image Understanding, 2006
- View Invariance for Human Action RecognitionInternational Journal of Computer Vision, 2006
- On Space-Time Interest PointsInternational Journal of Computer Vision, 2005
- The recognition of human movement using temporal templatesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2001
- Multivariant technique for multiclass pattern recognitionApplied Optics, 1980