Intention Action Anticipation Model with Guide-Feedback Loop Mechanism
Preprint
Abstract
Anticipating human intention from videos has broad applications, such as automatic driving, robot assistive technology, and virtual reality. This study addresses the problem of intention action anticipation using egocentric video sequences to estimate actions that indicate human intention. We propose a Hierarchical Complete-Recent (HCR) information fusion model that makes full use of the features of the entire video sequence (i.e., complete features) and the features of the video tail sequence (i.e., recent features). The HCR model has two primary mechanisms. The Guide-Feedback Loop (GFL) mechanism is proposed to model the relation between one recent feature and one complete feature. Based on GFL, the MultiComplete-Recent Feature Aggregation (MCRFA) module is proposed to model the relation of one recent feature with multiscale complete features. Based on GFL and MCRFA, the HCR model can hierarchically explore the rich interrelationships between multiscale complete features and multiscale recent features. Through comparative and ablation experiments, we validate the effectiveness of our model on two well-known public datasets: EPIC-Kitchens and EGTEA Gaze+.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Towards an autonomous clinical decision support systemEngineering Applications of Artificial Intelligence, 2023
- Streaming egocentric action anticipation: An evaluation scheme and approachComputer Vision and Image Understanding, 2023
- Rethinking Learning Approaches for Long-Term Action Anticipation2022
- Ambiguousness-Aware State Evolution for Action PredictionIEEE Transactions on Circuits and Systems for Video Technology, 2022
- RR-Net: Relation Reasoning for End-to-End Human-Object Interaction DetectionIEEE Transactions on Circuits and Systems for Video Technology, 2021
- Understanding Atomic Hand-Object Interaction With Human IntentionIEEE Transactions on Circuits and Systems for Video Technology, 2021
- Recurrent Semantic Preserving Generation for Action PredictionIEEE Transactions on Circuits and Systems for Video Technology, 2020
- Anticipation in Human-Robot Cooperation: A Recurrent Neural Network Approach for Multiple Action Sequences Prediction2018
- Towards robust intention estimation based on object affordance enabling natural human-robot collaboration in assembly tasksProcedia CIRP, 2018
- Temporal Segment Networks: Towards Good Practices for Deep Action Recognition2016