Seowon Han

This study proposes a human-in-the-loop dynamic graph-based planning framework designed to elevate LLM-based Embodied Agents from simple tools to trustworthy collaborative partners. To achieve this, we address the trade-off between the structural rigidity of plan-centric approaches and the instability of reactive methods. The framework utilizes a Directed Acyclic Graph (DAG) with AND/OR nodes to ensure robustness while maintaining flexibility. Critically, the agent features an Automated Recovery Mechanism for self-correction and a Dynamic Modification Mechanism that employs Relevance Analysis to effectively translate human interventions (Switch, Add, Delete) into graph updates. Comparative experiments in Minecraft with 30 participants validated the method’s effectiveness. The proposed agent (Agent B) outperformed the reactive baseline (Agent A), reducing mission completion time by 9.3%. Notably, the agent demonstrated high instruction compliance and reduced user frustration by approximately 20%, leading to statistically higher satisfaction scores (PSSUQ). These results confirm that by ensuring planning robustness and responsiveness, the proposed framework successfully enables agents to function as trustworthy partners in complex environments.

Virtual reality offers ordinary users the ability to observe and interact with various abstract or concrete objects visualized in a three-dimensional space from different angles. Users can manipulate, transform, or reconstruct these objects similarly to how they might in a real environment. Manipulating objects in virtual reality is not as effortless as in the real world, due to the lack of sensory feedback and limited input freedom. However, it also offers new advantages that the real world cannot provide, such as the ability to easily select and control remote objects and the support of various auxiliary user interfaces. In particular, when it is necessary to alternately manipulate objects of various sizes, scaling the user’s avatar symmetrically allows for more effective manipulation than in the real world. However, manual scaling interfaces can be cumbersome and may induce dizziness. This study proposes an interaction technique that allows users to conveniently manipulate objects of various sizes without manual scale adjustment, by automatically and instantly adjusting the scale factor according to the size of the selected object and its adjacent objects. To compensate for the change in scale, we also implement a position correction mechanism that adjusts the user’s position in the virtual environment. Preliminary experiments with a small group of participants confirmed that automatic scale adjustment produces significant effects. Based on the feedback from these experiments, a more refined distance calculation method and the timing for scale adjustment were derived. In the main experiment with 14 participants, it was confirmed that the automatic scale adjustment method proposed in this study led to higher accuracy and lower discomfort in task completion compared to the conventional manual scale adjustment method. We expect that the results of this study will effectively contribute to the creation of virtual reality content that requires interaction with objects of various sizes in the future.