Learning to Track for Spatio-Temporal Action Localization

Authors:
  • PWPhilippe Weinzaepfel
    Philippe Weinzaepfel
  • ZHZaid Harchaoui
    Zaid Harchaoui
  • CSCordelia Schmid
    Cordelia Schmid
Publisher website
Google Scholar
Abstract
We propose an effective approach for spatio-temporal action localization in realistic videos. The approach first detects proposals at the frame-level and scores them with a combination of static and motion CNN features. It then tracks high-scoring proposals throughout the video using a tracking-by-detection approach. Our tracker relies simultaneously on instance-level and class-level detectors. The tracks are scored using a spatio-temporal motion histogram, a descriptor at the track level, in combination with the CNN features. Finally, we perform temporal localization of the action using a sliding-window approach at the track level. We present experimental results for spatio-temporal localization on the UCF-Sports, J-HMDB and UCF-101 action localization datasets, where our approach outperforms the state of the art with a margin of 15%, 7% and 12% respectively in mAP.

This publication has 28 references indexed in Scilit: