Action Recognition Network Using Stacked Short-Term Deep Features and Bidirectional Moving Average
Open Access
- 16 June 2021
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 11 (12) , 5563
Abstract
Action recognition requires the accurate analysis of action elements in the form of a video clip and a properly ordered sequence of the elements. To solve the two sub-problems, it is necessary to learn both spatio-temporal information and the temporal relationship between different action elements. Existing convolutional neural network (CNN)-based action recognition methods have focused on learning only spatial or temporal information without considering the temporal relation between action elements. In this paper, we create short-term pixel-difference images from the input video, and take the difference images as an input to a bidirectional exponential moving average sub-network to analyze the action elements and their temporal relations. The proposed method consists of: (i) generation of RGB and differential images, (ii) extraction of deep feature maps using an image classification sub-network, (iii) weight assignment to extracted feature maps using a bidirectional, exponential, moving average sub-network, and (iv) late fusion with a three-dimensional convolutional (C3D) sub-network to improve the accuracy of action recognition. Experimental results show that the proposed method achieves a higher performance level than existing baseline methods. In addition, the proposed action recognition network takes only 0.075 seconds per action class, which guarantees various high-speed or real-time applications, such as abnormal action classification, human–computer interaction, and intelligent visual surveillance.Keywords
Funding Information
- Institute for Information & Communica248 tions Technology Promotion (2017-0-00250)
- ICT R & D program of MSIP/IITP (2014-0-00077)
- Chung-Ang University Research Scholarship Grants (2019)
This publication has 13 references indexed in Scilit:
- Action Recognition Based on the Fusion of Graph Convolutional Networks with High Order FeaturesApplied Sciences, 2020
- Semi-CNN Architecture for Effective Spatio-Temporal Learning in Action RecognitionApplied Sciences, 2020
- Human Action Recognition Based on Foreground Trajectory and Motion Difference DescriptorsApplied Sciences, 2019
- Densely Connected Convolutional Networks2017
- ImageNet classification with deep convolutional neural networksCommunications of the ACM, 2017
- Shuffle and Learn: Unsupervised Learning Using Temporal Order Verification2016
- Learning Spatiotemporal Features with 3D Convolutional Networks2015
- Large-Scale Video Classification with Convolutional Neural Networks2014
- 3D Convolutional Neural Networks for Human Action RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2012
- Lucas-Kanade 20 Years On: A Unifying FrameworkInternational Journal of Computer Vision, 2004