The Augmented Intelligence Perspective on Human-in-the-Loop Reinforcement Learning: Review, Concept Designs, and Future Directions
- 18 October 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Human-Machine Systems
- Vol. 54 (6) , 762-777
Abstract
Augmented intelligence (AuI) is a concept that combines human intelligence (HI) and artificial intelligence (AI) to leverage their respective strengths. While AI typically aims to replace humans, AuI integrates humans into machines, recognizing their irreplaceable role. Meanwhile, human-in-the-loop reinforcement learning (HITL-RL) is a semisupervised algorithm that integrates humans into the traditional reinforcement learning (RL) algorithm, enabling autonomous agents to gather inputs from both humans and environments, learn, and select optimal actions across various environments. Both AuI and HITL-RL are still in their infancy. Based on AuI, we propose and investigate three separate concept designs for HITL-RL: HI-AI , AI-HI , and parallel-HI-and-AI approaches, each differing in the order of HI and AI involvement in decision making. The literature on AuI and HITL-RL offers insights into integrating HI into existing concept designs. A preliminary study in an Atari game offers insights for future research directions. Simulation results show that human involvement maintains RL convergence and improves system stability, while achieving approximately similar average scores to traditional $Q$ -learning in the game. Future research directions are proposed to encourage further investigation in this area.Keywords
Funding Information
- Ministry of Higher Education, Malaysia
- Fundamental Research Grant Scheme (FRGS/1/2022/ICT02/UTAR/01/2)
This publication has 79 references indexed in Scilit:
- A survey of human-in-the-loop for machine learningFuture Generation Computer Systems, 2022
- AR-assisted digital twin-enabled robot collaborative manufacturing system with human-in-the-loopRobotics and Computer-Integrated Manufacturing, 2022
- Augmented Intelligence: Surveys of Literature and Expert Opinion to Understand Relations Between Human Intelligence and Artificial IntelligenceIEEE Access, 2021
- Human-in-the-Loop Learning Methods Toward Safe DL-Based Autonomous Systems: A Review2021
- Taking both sides: seeking symbiosis between intelligent prostheses and human motor control during locomotionCurrent Opinion in Biomedical Engineering, 2021
- Personalization of Hearing Aid Compression by Human-in-the-Loop Deep Reinforcement LearningIEEE Access, 2020
- A Review on Interactive Reinforcement Learning From Human Social FeedbackIEEE Access, 2020
- Toward Human-in-the-Loop PID Control Based on CACLA Reinforcement Learning2019
- Learning assistive strategies for exoskeleton robots from user-robot physical interactionPattern Recognition Letters, 2017
- Teachable robots: Understanding human teaching behavior to build more effective robot learnersArtificial Intelligence, 2008