Deep Multimodal Features for Movie Genre and Interestingness Prediction
Authors:
- 1 September 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Abstract
In this paper, we propose a multimodal framework for video segment interestingness prediction based on the genre and affective impact of movie content. We hypothesize that the emotional characteristic and impact of a video infer its genre, which can in turn be a factor for identifying the perceived interestingness of a particular video segment (shot) within the entire media. Our proposed approach is based on audio-visual deep features for perceptual content analysis. The multimodal content is quantified in a mid-level representation which consists in describing each audio-visual segment as a distribution over various genres (action, drama, horror, romance, sci-fi for now). Some segment might be more characteristic of a media and therefore be more interesting than a segment containing content with a neutral genre. Having determined the genre of individual video segments, we trained a classifier to produce an interestingness factor which is then used to rank segments. We evaluate our approach on the MediaEval2017 Media Interestingness Prediction Task Dataset (PMIT). We demonstrate that our approach outperforms the existing video interestingness approaches on the PMIT dataset in terms of Mean Average Precision.Keywords
This publication has 22 references indexed in Scilit:
- Analyzing and Predicting GIF Interestingness2016
- A comprehensive study on mid-level representation and ensemble learning for emotional analysis of video materialMultimedia Tools and Applications, 2016
- The Quest for Visual Interest2015
- Review of Methods to Predict Social Image Interestingness and Memorability2015
- Long-term recurrent convolutional networks for visual recognition and description2015
- Video summarization by learning submodular mixtures of objectives2015
- Video Affective Content Analysis: A Survey of State-of-the-Art MethodsIEEE Transactions on Affective Computing, 2015
- Sentiment Flow for Video Interestingness Prediction2014
- Understanding Affective Content of Music Videos through Learned Representations2014
- Recent developments in openSMILE, the munich open-source multimedia feature extractor2013