PHI: Bridging Domain Shift in Long-Term Action Quality Assessment via Progressive Hierarchical Instruction
Preprint
Abstract
Long-term Action Quality Assessment (AQA) aims to evaluate the quantitative performance of actions in long videos. However, existing methods face challenges due to domain shifts between the pre-trained large-scale action recognition backbones and the specific AQA task, thereby hindering their performance. This arises since fine-tuning resource-intensive backbones on small AQA datasets is impractical. We address this by identifying two levels of domain shift: task-level, regarding differences in task objectives, and feature-level, regarding differences in important features. For feature-level shifts, which are more detrimental, we propose Progressive Hierarchical Instruction (PHI) with two strategies. First, Gap Minimization Flow (GMF) leverages flow matching to progressively learn a fast flow path that reduces the domain gap between initial and desired features across shallow to deep layers. Additionally, a temporally-enhanced attention module captures long-range dependencies essential for AQA. Second, List-wise Contrastive Regularization (LCR) facilitates coarse-to-fine alignment by comprehensively comparing batch pairs to learn fine-grained cues while mitigating domain shift. Integrating these modules, PHI offers an effective solution. Experiments demonstrate that PHI achieves state-of-the-art performance on three representative long-term AQA datasets, proving its superiority in addressing the domain shift for long-term AQA.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- On the design fundamentals of diffusion models: A surveyPattern Recognition, 2025
- FineCausal: A Causal-Based Framework for Interpretable Fine-Grained Action Quality Assessment2025
- Fg-T2M++: LLMs-Augmented Fine-Grained Text Driven Human Motion GenerationInternational Journal of Computer Vision, 2025
- Two-path target-aware contrastive regression for action quality assessmentInformation Sciences, 2024
- Multimodal Action Quality AssessmentIEEE Transactions on Image Processing, 2024
- Fine-Grained Spatio-Temporal Parsing Network for Action Quality AssessmentIEEE Transactions on Image Processing, 2023
- Learning Semantics-Guided Representations for Scoring Figure SkatingIEEE Transactions on Multimedia, 2023
- A Contrastive Learning Network for Performance Metric and Assessment of Physical Rehabilitation ExercisesIEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023
- A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile DermatomyositisIEEE Transactions on Visualization and Computer Graphics, 2023
- Normalizing Flows: An Introduction and Review of Current MethodsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2020