Hierarchical Planning and Policy Shaping Shared Autonomy for Articulated Robots
Preprint
Abstract
In this work, we propose a novel shared autonomy framework to operate articulated robots. We provide strategies to design both the task-oriented hierarchical planning and policy shaping algorithms for efficient human-robot interactions in context-aware operation of articulated robots. Our framework for interplay between the human and the autonomy, as the participating agents in the system, is particularly influenced by the ideas from multi-agent systems, game theory, and theory of mind for a sliding level of autonomy. We formulate the sequential hierarchical human-in-the-loop decision making process by extending MDPs and Options framework to shared autonomy, and make use of deep RL techniques to train an uncertainty-aware shared autonomy policy. To fine-tune the formulation to a human, we use history of the system states, human actions, and their error with respect to a surrogate optimal model to encode human's internal state embeddings, beyond the designed values, by using conditional VAEs. We showcase the effectiveness of our formulation for different human skill levels and degrees of cooperativeness by using a case study of a feller-buncher machine in the challenging tasks of timber harvesting. Our framework is successful in providing a sliding level of autonomy from fully autonomous to fully manual, and is particularly successful in handling a noisy non-cooperative human agent in the loop. The proposed framework advances the state-of-the-art in shared autonomy for operating articulated robots, but can also be applied to other domains where autonomous operation is the ultimate goal.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Control for Societal-Scale Challenges: Road Map 2030IEEE Control Systems, 2024
- Local Autonomy-Based Haptic-Robot Interaction With Dual-Proxy ModelIEEE Transactions on Robotics, 2022
- Learning latent actions to control assistive robotsAutonomous Robots, 2021
- Hierarchical Reinforcement LearningACM Computing Surveys, 2021
- Deep Reinforcement Learning for Autonomous Driving: A SurveyIEEE Transactions on Intelligent Transportation Systems, 2021
- Shared Control Between Pilots and Autopilots: An Illustration of a Cyberphysical Human SystemIEEE Control Systems, 2020
- Hierarchical MPC for Robust Eco-Cooling of Connected and Automated Vehicles and Its Application to Electric Vehicle Battery Thermal ManagementIEEE Transactions on Control Systems Technology, 2020
- Predictive shared steering control for driver override in automated driving: A simulator studyTransportation Research Part F: Traffic Psychology and Behaviour, 2019
- Imitation-based control of automated ore excavator: improvement of autonomous excavation database quality using clustering and association analysis processesAdvanced Robotics, 2017
- Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learningArtificial Intelligence, 1999