Audio-Visual Self-Supervision for Frame-Level Player-wise Offensive Shot Detection in Table Tennis Matches
Open Access
- 28 October 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 27-33
Abstract
Understanding decision-making processes is informative for strategic planning. Aiming to understand human risk-taking behavior in decision-making, we investigate the possibility of classifying whether a shot is offensive or not, targeting table tennis videos. We define the problem in a multi-task setting: detecting shots with frame-level precision while classifying shot offensiveness, and, as an optional task, predicting which player made the shot. We use commercial table tennis videos for target analysis and propose audio-visual self-supervised training, leveraging web videos with similar camera views. Our local contrastive loss encourages the model to learn frame-wise action locality collaboratively with traditional segment-wise contrastive loss, which we call global contrastive loss. Experimental results proved that the collaboration of two contrastive losses boosts the prediction performance.Keywords
Funding Information
- JSPS KAKENHI (21K18568)
- JST PRESTO (JPMJPR22C6)
This publication has 20 references indexed in Scilit:
- Table Tennis Stroke Recognition Using Two-Dimensional Human Pose Estimation2021
- Progressive Relation Learning for Group Activity Recognition2020
- TTNet: Real-time temporal and spatial video analysis of table tennis2020
- Refining Joint Locations for Human Pose Tracking in Sports Videos2019
- The Sound of Pixels2018
- Unsupervised Learning of Long-Term Motion Dynamics for Videos2017
- Shuffle and Learn: Unsupervised Learning Using Temporal Order Verification2016
- Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles2016
- Athlete Pose Estimation from Monocular TV Sports Footage2013
- Reconstruction of 3D Trajectories for Performance Analysis in Table Tennis2013