QUB-PHEO: A Visual-Based Dyadic Multi-View Dataset for Intention Inference in Collaborative Assembly
Open Access
- 23 October 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 157050-157066
Abstract
QUB-PHEO introduces a visual-based, dyadic dataset with the potential of advancing human-robot interaction (HRI) research in assembly operations and intention inference. This dataset captures rich multimodal interactions between two participants, one acting as a ’robot surrogate,’ across a variety of assembly tasks that are further broken down into 36 distinct subtasks. With rich visual annotations—facial landmarks, gaze, hand movements, object localization, and more—for 70 participants, QUB-PHEO offers two versions: full video data for 50 participants and visual cues for all 70. Designed to improve machine learning models for HRI, QUB-PHEO enables deeper analysis of subtle interaction cues and intentions, promising contributions to the field. The dataset is available at https://github.com/exponentialR/QUB-PHEO subject to an End-User License Agreement (EULA).Keywords
Funding Information
- Engineering and Physical Sciences and Research Council (EPSRC), U.K
This publication has 43 references indexed in Scilit:
- Hand-Eye-Object Tracking for Human Intention InferenceIFAC-PapersOnLine, 2022
- Gaze Based Implicit Intention Inference with Historical Information of Visual Attention for Human-Robot Interaction2021
- Multi-ship encounter situation adaptive understanding by individual navigation intention inferenceOcean Engineering, 2021
- M-MS: A Multi-Modal Synchrony Dataset to Explore Dyadic Interaction in ASD2020
- How Robots Influence Humans: A Survey of Nonverbal Communication in Social Human–Robot InteractionInternational Journal of Social Robotics, 2019
- Action Alignment from Gaze Cues in Human-Human and Human-Robot Interaction2019
- Don’t Stare at Me: The Impact of a Humanoid Robot’s Gaze upon Trust During a Cooperative Human–Robot Visual TaskInternational Journal of Social Robotics, 2017
- The JESTKOD database: an affective multimodal database of dyadic interactionsLanguage Resources and Evaluation, 2016
- The USC CreativeIT database of multimodal dyadic interactions: from speech and full body motion capture to continuous emotional annotationsLanguage Resources and Evaluation, 2015
- Two Faces of Human–Robot Interaction: Field and Service Robots2014