LSTM Pose Machines
Authors:
- Yue LuoYLYue Luo
- Jimmy RenJRJimmy Ren
- Zhouxia WangZWZhouxia Wang
- Wenxiu SunWSWenxiu Sun
- Jinshan PanJPJinshan Pan
- Jianbo LiuJLJianbo Liu
- Jiahao PangJPJiahao Pang
- Liang LinLLLiang Lin
- 1 June 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5207-5215
Abstract
We observed that recent state-of-the-art results on single image human pose estimation were achieved by multistage Convolution Neural Networks (CNN). Notwithstanding the superior performance on static images, the application of these models on videos is not only computationally intensive, it also suffers from performance degeneration and flicking. Such suboptimal results are mainly attributed to the inability of imposing sequential geometric consistency, handling severe image quality degradation (e.g. motion blur and occlusion) as well as the inability of capturing the temporal correlation among video frames. In this paper, we proposed a novel recurrent network to tackle these problems. We showed that if we were to impose the weight sharing scheme to the multi-stage CNN, it could be re-written as a Recurrent Neural Network (RNN). This property decouples the relationship among multiple network stages and results in significantly faster speed in invoking the network for videos. It also enables the adoption of Long Short-Term Memory (LSTM) units between video frames. We found such memory augmented RNN is very effective in imposing geometric consistency among frames. It also well handles input quality degradation in videos while successfully stabilizes the sequential outputs. The experiments showed that our approach significantly outperformed current state-of-the-art methods on two large-scale video pose estimation benchmarks. We also explored the memory cells inside the LSTM and provided insights on why such mechanism would benefit the prediction for video-based pose estimations 1 .Keywords
This publication has 24 references indexed in Scilit:
- Flowing ConvNets for Human Pose Estimation in Videos2015
- Understanding deep image representations by inverting them2015
- Joint action recognition and pose estimation from video2015
- DeepPose: Human Pose Estimation via Deep Neural Networks2014
- Towards Understanding Action Recognition2013
- Poselet Conditioned Pictorial Structures2013
- N-best maximal decoders for part models2011
- Learning effective human pose estimation from inaccurate annotation2011
- Pictorial Structures for Object RecognitionInternational Journal of Computer Vision, 2005
- Long Short-Term MemoryNeural Computation, 1997