RefineTAD: Learning Proposal-free Refinement for Temporal Action Detection
Open Access
- 26 October 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 135-143
Abstract
Temporal action detection (TAD) aims to localize the start and end frames of actions in untrimmed videos, which is a challenging task due to the similarity of adjacent frames and the ambiguity of action boundaries. Previous methods often generate coarse proposals first and then perform proposal-based refinement, which is coupled with prior action detectors and leads to proposal-oriented offsets. However, this paradigm increases the training difficulty of the TAD model and is heavily influenced by the quantity and quality of the proposals. To address the above issues, we decouple the refinement process from conventional TAD methods and propose a learnable, proposal-free refinement method for fine boundary localization, named RefineTAD. We first propose a multi-level refinement module to generate multi-scale boundary offsets, score offsets and boundary-aware probability at each time point based on the feature pyramid. Then, we propose an offset focusing strategy to progressively refine the predicted results of TAD models in a coarse-to-fine manner with our multi-scale offsets. We perform extensive experiments on three challenging datasets and demonstrate that our RefineTAD significantly improves the state-of-the-art TAD methods with minimal computational overhead.Keywords
Funding Information
- National Natural Science Foundation of China (62276129)
- Natural Science Foundation of Jiangsu Province (BK20220890)
This publication has 29 references indexed in Scilit:
- Step-by-step Erasion, One-by-one Collection2018
- BSN: Boundary Sensitive Network for Temporal Action Proposal Generation2018
- Precise Temporal Action Localization by Evolving Temporal Proposals2018
- Focal Loss for Dense Object Detection2017
- Soft-NMS — Improving Object Detection with One Line of Code2017
- Xception: Deep Learning with Depthwise Separable Convolutions2017
- UntrimmedNets for Weakly Supervised Action Recognition and Detection2017
- Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset2017
- Fast R-CNN2015
- ActivityNet: A large-scale video benchmark for human activity understanding2015