Abstract
Immersive virtual and mixed reality systems continuously record head and hand/controller motion, yet these traces are commonly reduced to aggregate task outcomes or retained only for technical purposes. This study investigates whether coordinate-aware motion telemetry can instead serve as behavioral process data for understanding how interaction unfolds over time. We harmonized telemetry from four immersive contexts: full-scale VR assembly (FAST; 108 participants, 216 sessions), physically grounded MR/VR driving in a real vehicle (40 participants, 80 paired runs), educational cartography in GeoConnect (206 performance-linked stages from 35 contributing sessions), and virtual logic-circuit construction (177 sessions). The analysis combined kinematic, head-relative, bilateral, temporal-state, and performance-grounded descriptors while explicitly preserving the coordinate semantics of each task. FAST exhibited a robust bimanual-centered temporal profile, with 80.6\% of analyzed windows classified as bimanual. Repeated exposure was associated with shorter completion time and reduced head and left-hand movement, but increased right-hand specialization. In the driving context, world-space HMD paths exceeded 3~km despite participants remaining physically seated, demonstrating that mathematically valid trajectories can be behaviorally misleading when coordinate context is ignored. In GeoConnect, motion was associated with task duration, attempts, errors, and hesitation under participant-clustered inference; HMD path, for example, was strongly associated with stage duration ($\rho=.748$, 95\% CI [0.674, 0.814]). These findings support coordinate-aware motion telemetry as a complementary source of behavioral evidence for immersive HCI, while emphasizing that inferred motion states must remain grounded in task context rather than treated as universal cognitive labels.

This publication has 13 references indexed in Scilit: