Evaluating Two-Stream CNN for Video Classification
Authors:
Open Access
- 22 June 2015
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 435-442
Abstract
Videos contain very rich semantic information. Traditional hand-crafted features are known to be inadequate in analyzing complex video semantics. Inspired by the huge success of the deep learning methods in analyzing image, audio and text data, significant efforts are recently being devoted to the design of deep nets for video analytics. Among the many practical needs, classifying videos (or video clips) based on their major semantic categories (e.g.,"skiing") is useful in many applications. In this paper, we conduct an in-depth study to investigate important implementation options that may affect the performance of deep nets on video classification. Our evaluations are conducted on top of a recent two-stream convolutional neural network (CNN) pipeline, which uses both static frames and motion optical flows, and has demonstrated competitive performance against the state-of-the-art methods. In order to gain insights and to arrive at a practical guideline, many important options are studied, including network architectures, model fusion, learning parameters and the final prediction methods. Based on the evaluations, very competitive results are attained on two popular video classification benchmarks. We hope that the discussions and conclusions from this work can help researchers in related fields to quickly set up a good basis for further investigations along this very promising direction.Keywords
All Related Versions
Funding Information
- National 863 Program (2014AA015101)
- National Natural Science Foundation of China (61201387)
- Science and Technology Commission of Shanghai Municipality (13PJ1400400, 13511504503)
This publication has 22 references indexed in Scilit:
- Translating Videos to Natural Language Using Deep Recurrent Neural Networks2015
- Deep learning in neural networks: An overviewNeural Networks, 2015
- Exploring Inter-feature and Inter-class Relationships with Deep Neural Networks for Video Classification2014
- A Convolutional Neural Network for Modelling Sentences2014
- Action Recognition with Improved Trajectories2013
- Discovering joint audio–visual codewords for video event detectionMachine Vision and Applications, 2013
- Representation Learning: A Review and New PerspectivesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Deep Hierarchies in the Primate Visual Cortex: What Can We Learn for Computer Vision?IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012
- Early versus late fusion in semantic video analysis2005
- Separate visual pathways for perception and actionTrends in Neurosciences, 1992