ProposalVLAD with Proposal-Intra Exploring for Temporal Action Proposal Generation
Open Access
- 25 February 2023
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 19 (3) , 1-18
Abstract
Temporal action proposal generation aims to localize temporal segments of human activities in videos. Current boundary-based proposal generation methods can generate proposals with precise boundary, but often suffer from the inferior quality of confidence scores used for proposal retrieving. In this paper, we propose an effective and end-to-end action proposal generation method, named ProposalVLAD with Proposal-Intra Exploring Network (PVPI-Net). We first propose a ProposalVLAD module to dynamically generate global features of the entire video, then we combine the global features and proposal local features to generate the final feature representations for all candidate proposals. Then, we design a novel Proposal-Intra Loss function (PI-Loss) to generate more reliable proposal confidence scores. Extensive experiments on large-scale and challenging datasets demonstrate the effectiveness of our proposed method. Experimental results show that our PVPI-Net achieves significant improvements on two benchmark datasets (i.e., THUMOS’14 and ActivityNet-1.3), and sets new records for temporal action detection task.Keywords
This publication has 8 references indexed in Scilit:
- When Did It Happen? Duration-informed Temporal Localization of Narrated Actions in VlogsACM Transactions on Multimedia Computing, Communications, and Applications, 2022
- Progressive Meta-Learning With CurriculumIEEE Transactions on Circuits and Systems for Video Technology, 2022
- AMANet: Adaptive Multi-Path Aggregation for Learning Human 2D-3D CorrespondencesIEEE Transactions on Multimedia, 2021
- Proposal Complementary Action DetectionACM Transactions on Multimedia Computing, Communications, and Applications, 2020
- Action Recognition Using Form and Motion ModalitiesACM Transactions on Multimedia Computing, Communications, and Applications, 2020
- Moving Foreground-Aware Visual Attention and Key Volume Mining for Human Action RecognitionACM Transactions on Multimedia Computing, Communications, and Applications, 2019
- Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks2017
- Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset2017