Audio-Based Semantic Concept Classification for Consumer Video
Authors:
- 20 October 2009
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech, and Language Processing
- Vol. 18 (6) , 1406-1416
Abstract
This paper presents a novel method for automatically classifying consumer video clips based on their soundtracks. We use a set of 25 overlapping semantic classes, chosen for their usefulness to users, viability of automatic detection and of annotator labeling, and sufficiency of representation in available video collections. A set of 1873 videos from real users has been annotated with these concepts. Starting with a basic representation of each video clip as a sequence of mel-frequency cepstral coefficient (MFCC) frames, we experiment with three clip-level representations: single Gaussian modeling, Gaussian mixture modeling, and probabilistic latent semantic analysis of a Gaussian component histogram. Using such summary features, we produce support vector machine (SVM) classifiers based on the Kullback-Leibler, Bhattacharyya, or Mahalanobis distance measures. Quantitative evaluation shows that our approaches are effective for detecting interesting concepts in a large collection of real-world consumer video clips.Keywords
This publication has 20 references indexed in Scilit:
- A Web-Based Game for Collecting Music MetadataJournal of New Music Research, 2008
- Large-scale multimodal semantic concept detection for consumer video2007
- Unveiling Music Structure via PLSA Similarity Fusion2007
- Classifying User Environment for Mobile Applications using Linear Autoencoding of Ambient Audio2006
- Acoustic environment classificationACM Transactions on Speech and Language Processing, 2006
- Audio-based context recognitionIEEE Transactions on Audio, Speech, and Language Processing, 2005
- Minimal-impact audio-based personal archives2004
- Speech/music segmentation using entropy and dynamism features in a HMM classification frameworkSpeech Communication, 2003
- Content-based audio classification and retrieval by support vector machinesIEEE Transactions on Neural Networks, 2003
- Construction and evaluation of a robust multifeature speech/music discriminator2002