Visual Event Recognition in News Video using Kernel Methods with Multi-Level Temporal Alignment
- 1 June 2007
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636919,p. 1-8
Abstract
In this work, we systematically study the problem of visual event recognition in unconstrained news video sequences. We adopt the discriminative kernel-based method for which video clip similarity plays an important role. First, we represent a video clip as a bag of orderless descriptors extracted from all of the constituent frames and apply Earth mover's distance (EMD) to integrate similarities among frames from two clips. Observing that a video clip is usually comprised of multiple sub-clips corresponding to event evolution over time, we further build a multilevel temporal pyramid. At each pyramid level, we integrate the information from different sub-clips with Integer-value-constrained EMD to explicitly align the sub-clips. By fusing the information from the different pyramid levels, we develop temporally aligned pyramid matching (TAPM) for measuring video similarity. We conduct comprehensive experiments on the Trecvid 2005 corpus, which contains more than 6,800 clips. Our experiments demonstrate that 1) the TAPM multi-level method clearly outperforms single-level EMD, and 2) single-level EMD outperforms by a large margin (43.0% in Mean Average Precision) basic detection methods that use only a single key-frame. Extensive analysis of the results also reveals an intuitive interpretation of subclip alignment at different levels.Keywords
This publication has 14 references indexed in Scilit:
- Large-Scale Concept Ontology for MultimediaIEEE MultiMedia, 2006
- The Function Space of an Activity2006
- Semi-Supervised Adapted HMMs for Unusual Event Detection2005
- Detecting irregularities in images and in video2005
- Object labelling from human action recognition2004
- Video Google: a text retrieval approach to object matching in videos2003
- Space-time interest points2003
- A Bayesian computer vision system for modeling human interactionsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000
- Data clusteringACM Computing Surveys, 1999
- Object recognition from local scale-invariant features1999