Heterogeneous Hyper-Graph Neural Networks for Context-aware Human Activity Recognition
Preprint
Abstract
Context-aware Human Activity Recognition (CHAR) is challenging due to the need to recognize the user's current activity from signals that vary significantly with contextual factors such as phone placements and the varied styles with which different users perform the same activity. In this paper, we argue that context-aware activity visit patterns in realistic in-the-wild data can equivocally be considered as a general graph representation learning task. We posit that exploiting underlying graphical patterns in CHAR data can improve CHAR task performance and representation learning. Building on the intuition that certain activities are frequently performed with the phone placed in certain positions, we focus on the context-aware human activity problem of recognizing the tuple. We demonstrate that CHAR data has an underlying graph structure that can be viewed as a heterogenous hypergraph that has multiple types of nodes and hyperedges (an edge connecting more than two nodes). Subsequently, learning representations becomes a graph node representation learning problem. After task transformation, we further propose a novel Heterogeneous HyperGraph Neural Network architecture for Context-aware Human Activity Recognition (HHGNN-CHAR), with three types of heterogeneous nodes (user, phone placement, and activity). Connections between all types of nodes are represented by hyperedges. Rigorous evaluation demonstrated that on an unscripted, in-the-wild CHAR dataset, our proposed framework significantly outperforms state-of-the-art (SOTA) baselines including CHAR models that do not exploit graphs, and GNN variants that do not incorporate heterogeneous nodes or hyperedges with overall improvements 14.04% on Matthews Correlation Coefficient (MCC) and 7.01% on Macro F1 scores.Keywords
All Related Versions
- Version 1, 2024-09-26, ArXiv
- Published version: , 350.
This publication has 8 references indexed in Scilit:
- Hypergraph convolution and hypergraph attentionPattern Recognition, 2021
- Moodable: On feasibility of instantaneous depression assessment using machine learning on voice samples with retrospectively harvested smartphone and social media dataSmart Health, 2020
- Revisiting User Mobility and Social Relationships in LBSNs: A Hypergraph Embedding Approach2019
- Recognizing Detailed Human Context in the Wild from Smartphones and SmartwatchesIEEE Pervasive Computing, 2017
- Evaluating the specifications of built-in accelerometers in smartphones on fall detection performanceInstrumentation Science & Technology, 2017
- A survey of energy-efficient context recognition systems using wearable sensors for healthcare applicationsPervasive and Mobile Computing, 2017
- Deploying personalized mobile services in an agent-based environmentExpert Systems with Applications, 2007
- A Conceptual Framework and a Toolkit for Supporting the Rapid Prototyping of Context-Aware ApplicationsHuman–Computer Interaction, 2001