Learn to Adapt: A Policy for History-Based Online Adaptation
Open Access
- 2 April 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computational Intelligence
- Vol. 9 (6) , 4026-4036
Abstract
Although today's reinforcement learning-based control agents operate effectively in ideal situations, their performance can significantly decrease under changing conditions. This paper addresses this limitation by proposing an experience-based online adaptation framework. It is designed to enable agents to adjust their policies to differing domains and scenarios by leveraging the information embedded in past state-action transitions. The method extracts these extra features via a history-based adaptor module and introduces them as supplementary input for the agents. It also contains a state encoder network, improving the representation capabilities and supporting the adaptation process. Furthermore, the introduced modules can replace hard-to-obtain information during deployment (e.g., sensor or domain-specific data), which facilitates real-world applications. To emphasize the contribution of our work, we evaluate the solution within highly dynamic environments where the changing physical parameters necessitate diverse control strategies. Our approach significantly improves conventional algorithms and results in comparable performance with agents receiving privileged information.Keywords
Funding Information
- European Union through the National Laboratory for Autonomous Systems (RRF-2.3.1-21-2022-00002)
- Ministry of Innovation and Technology of Hungary
- Nemzeti Kutatási, Fejlesztési és Innovaciós Alap
- Ministry of Culture and Innovation of Hungary from the National Research (KDP-2023, KDP-2021)
This publication has 42 references indexed in Scilit:
- A deep reinforcement learning algorithm to control a two-wheeled scooter with a humanoid robotEngineering Applications of Artificial Intelligence, 2023
- Champion-level drone racing using deep reinforcement learningNature, 2023
- Transferring policy of deep reinforcement learning from simulation to reality for roboticsNature Machine Intelligence, 2022
- Connected automated vehicle cooperative control with a deep reinforcement learning approach in a mixed traffic environmentTransportation Research Part C: Emerging Technologies, 2021
- Grounded action transformation for sim-to-real reinforcement learningMachine Learning, 2021
- Combining system identification with reinforcement learning-based MPCIFAC-PapersOnLine, 2020
- Continuous Control with Deep Reinforcement Learning for Mobile Robot Navigation2019
- Position/force control of robot manipulators using reinforcement learningIndustrial Robot: the international journal of robotics research and application, 2019
- Domain-Adversarial Training of Neural Networks2017
- Robust trajectory optimization under frictional contact with iterative learningAutonomous Robots, 2017