Object-Based Temporal Segment Relational Network for Activity Recognition
Authors:
- Victor H. C. MeloVMVictor H. C. Melo
- Jesimon Barreto SantosJSJesimon Barreto Santos
- Carlos CaetanoCCCarlos Caetano
- Jessica SenaJSJessica Sena
- Otávio Augusto Bizetto PenattiOPOtávio Augusto Bizetto Penatti
- William Robson SchwartzWSWilliam Robson Schwartz
- 1 October 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 103-109
Abstract
Video understanding is the next frontier of computer vision, in which activity recognition plays a major role. Despite the recent improvements in holistic activity recognition, further researching part-based models such as context may allow us to better understand what is important for activities and thus improve our current activity recognition models. This work tackles contextual cues obtained from object detections, in which we posit that objects relevant to an action are related to its spatial arrangement regarding an agent. Based on that, we propose Egocentric Pyramid to encode such spatial relationships. We further extend it by proposing a data-centric approach named Temporal Segment Relational Network (TSRN). Our experiments give support to the hypothesis that object spatiality provides an important clue to activity recognition. In addition, our data-centric approach shows that besides such spatial features, there may be other important information that further enhances the object-based activity recognition, such as co-occurrence, relative size, and temporal information.Keywords
This publication has 10 references indexed in Scilit:
- The “Something Something” Video Database for Learning and Evaluating Visual Common Sense2017
- Densely Connected Convolutional Networks2017
- Exploiting Feature and Class Relationships in Video Categorization with Regularized Deep Neural NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Deep Residual Learning for Image Recognition2016
- P-CNN: Pose-Based CNN Features for Action Recognition2015
- What do 15,000 object categories tell us about classifying and localizing actions?2015
- Visual word spatial arrangement for image retrieval and classificationPattern Recognition, 2014
- Action Recognition with Improved Trajectories2013
- Spatio-temporal Human-Object Interactions for Action Recognition in Videos2013
- Histograms of Oriented Gradients for Human Detection2005