Actor-Action Semantic Segmentation with Grouping Process Models
Authors:
- Chenliang XuCXChenliang Xu
- Jason J. CorsoJCJason J. Corso
- 1 June 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3083-3092
Abstract
Actor-action semantic segmentation made an important step toward advanced video understanding: what action is happening, who is performing the action, and where is the action happening in space-time. Current methods based on layered CRFs for this problem are local and unable to capture the long-ranging interactions of video parts. We propose a new model that combines the labeling CRF with a supervoxel hierarchy, where supervoxels at various scales provide cues for possible groupings of nodes in the CRF to encourage adaptive and long-ranging interactions. The new model defines a dynamic and continuous process of information exchange: the CRF influences what supervoxels in the hierarchy are active, and these active supervoxels, in turn, affect the connectivities in the CRF, we hence call it a grouping process model. By further incorporating the video-level recognition, the proposed method achieves a large margin of 60% relative improvement over the state of the art on the recent A2D large-scale video labeling dataset, which demonstrates the effectiveness of our modeling.Keywords
All Related Versions
This publication has 32 references indexed in Scilit:
- Segmentation of Moving Objects by Long Term Video AnalysisIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- A Video Representation Using Temporal Superpixels2013
- Spatiotemporal Deformable Part Models for Action Detection2013
- Dense Trajectories and Motion Boundary Descriptors for Action RecognitionInternational Journal of Computer Vision, 2013
- Fast Approximate Energy Minimization with Label CostsInternational Journal of Computer Vision, 2011
- TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and ContextInternational Journal of Computer Vision, 2007
- On Space-Time Interest PointsInternational Journal of Computer Vision, 2005
- An experimental comparison of min-cut/max- flow algorithms for energy minimization in visionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2004
- What energy functions can be minimized via graph cuts?IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004
- Fast approximate energy minimization via graph cutsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2001