AVA: A Video Dataset of Spatio-Temporally Localized Atomic Visual Actions
Authors:
- Chunhui GuCGChunhui Gu
- Chen SunCSChen Sun
- David A. RossDRDavid A. Ross
- Carl VondrickCVCarl Vondrick
- Caroline PantofaruCPCaroline Pantofaru
- Yeqing LiYLYeqing Li
- Sudheendra VijayanarasimhanSVSudheendra Vijayanarasimhan
- George TodericiGTGeorge Toderici
- Susanna RiccoSRSusanna Ricco
- Rahul SukthankarRSRahul Sukthankar
- Cordelia SchmidCSCordelia Schmid
- Jitendra MalikJMJitendra Malik
Top Cited Papers
- 1 June 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6047-6056
Abstract
This paper introduces a video dataset of spatio-temporally localized Atomic Visual Actions (AVA). The AVA dataset densely annotates 80 atomic visual actions in 437 15-minute video clips, where actions are localized in space and time, resulting in 1.59M action labels with multiple labels per person occurring frequently. The key characteristics of our dataset are: (1) the definition of atomic visual actions, rather than composite actions; (2) precise spatio-temporal annotations with possibly multiple annotations for each person; (3) exhaustive annotation of these atomic actions over 15-minute video clips; (4) people temporally linked across consecutive segments; and (5) using movies to gather a varied set of action representations. This departs from existing datasets for spatio-temporal action recognition, which typically provide sparse annotations for composite actions in short video clips. AVA, with its realistic scene and action complexity, exposes the intrinsic difficulty of action recognition. To benchmark this, we present a novel approach for action localization that builds upon the current state-of-the-art methods, and demonstrates better performance on JHMDB and UCF101-24 categories. While setting a new state of the art on existing datasets, the overall results on AVA are low at 15.8% mAP, underscoring the need for developing new approaches for video understanding.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- HICO: A Benchmark for Recognizing Human-Object Interactions in Images2015
- Learning to Track for Spatio-Temporal Action Localization2015
- Finding action tubes2015
- ActivityNet: A large-scale video benchmark for human activity understanding2015
- The Pascal Visual Object Classes Challenge: A RetrospectiveInternational Journal of Computer Vision, 2014
- Large-Scale Video Classification with Convolutional Neural Networks2014
- Towards Understanding Action Recognition2013
- Actions as space-time shapes2005
- Recognizing human actions: a local SVM approach2004
- The Hungarian method for the assignment problemNaval Research Logistics Quarterly, 1955