Unsupervised Video Summarization Based on Spatiotemporal Semantic Graph and Enhanced Attention Mechanism
- 10 July 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Social Systems
- Vol. 12 (5) , 3751-3764
Abstract
Generative adversarial networks (GANs) have demonstrated potential in enhancing keyframe selection and video reconstruction via adversarial training among unsupervised approaches. Nevertheless, GANs struggle to encapsulate the intricate spatiotemporal dynamics in videos, which is essential for producing coherent and informative summaries. To address these challenges, we introduce an unsupervised video summarization framework that synergistically integrates temporal–spatial semantic graphs (TSSGraphs) with a bilinear additive attention (BAA) mechanism. TSSGraphs are designed to effectively model temporal and spatial relationships among video frames by combining temporal convolution and dynamic edge convolution, thereby extracting salient features while mitigating model complexity. The BAA mechanism enhances the framework’s ability to capture critical motion information by addressing feature sparsity and eliminating redundant parameters, ensuring robust attention to significant motion dynamics. Experimental assessments on the SumMe and TVSum benchmark datasets reveal that our method attains improvements of up to 4.0% and 3.3% in F-score, respectively, compared to current methodologies. Moreover, our system demonstrates diminished parameter overhead throughout training and inference stages, particularly excelling in contexts with significant motion content.Keywords
Funding Information
- National Natural Science Foundation of China (62372167)
- Natural Science Foundation of Hainan Province (2024JJ5097)
This publication has 51 references indexed in Scilit:
- Attention-guided multi-granularity fusion model for video summarizationExpert Systems with Applications, 2024
- Deep multi-scale pyramidal features network for supervised video summarizationExpert Systems with Applications, 2023
- Global-and-Local Relative Position Embedding for Unsupervised Video Summarization2020
- SumGraph: Video Summarization via Recursive Graph Modeling2020
- Static video summarization using multi-CNN with sparse autoencoder and random forest classifierSignal, Image and Video Processing, 2020
- Unsupervised Video Summarization via Attention-Driven Adversarial Learning2019
- Creating personalized video summaries via semantic event detectionJournal of Ambient Intelligence and Humanized Computing, 2018
- Video Summarization with Long Short-Term Memory2016
- Creating Summaries from User Videos2014
- Two-Frame Motion Estimation Based on Polynomial Expansion2003