Reward Training Wheels: Adaptive Auxiliary Rewards for Robotics Reinforcement Learning
Authors:
- 19 October 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 15262-15267
Abstract
Robotics Reinforcement Learning (RL) often relies on carefully engineered auxiliary rewards to supplement sparse primary learning objectives to compensate for the lack of large-scale, real-world, trial-and-error data. While these auxiliary rewards accelerate learning, they require significant engineering effort, may introduce human biases, and cannot adapt to the robot’s evolving capabilities during training. In this paper, we introduce Reward Training Wheels (RTW), a teacher-student framework that automates auxiliary reward adaptation for robotics RL. To be specific, the RTW teacher dynamically adjusts auxiliary reward weights based on the student’s evolving capabilities to determine which auxiliary reward aspects require more or less emphasis to improve the primary objective. We demonstrate RTW on two challenging robot tasks: navigation in highly constrained spaces and off-road vehicle mobility on vertically challenging terrain. In simulation, RTW outperforms expert-designed rewards by 2.35% in navigation success rate and improves off-road mobility performance by 122.62%, while achieving 35% and 3X faster training efficiency, respectively. Physical robot experiments further validate RTW’s effectiveness, achieving a perfect success rate (5/5 trials vs. 2/5 for expert-designed rewards) and improving vehicle stability with up to 47.4% reduction in orientation angles.Keywords
All Related Versions
Funding Information
- National Science Foundation
This publication has 16 references indexed in Scilit:
- II-NVM: Enhancing Map Accuracy and Consistency With Normal Vector-Assisted MappingIEEE Robotics and Automation Letters, 2025
- Autonomous Ground Navigation in Highly Constrained Spaces: Lessons Learned From the Third BARN Challenge at ICRA 2024 [Competitions]IEEE Robotics & Automation Magazine, 2024
- Toward Wheeled Mobility on Vertically Challenging Terrain: Platforms, Datasets, and Algorithms2024
- Autonomous Ground Navigation in Highly Constrained Spaces: Lessons Learned From the Second BARN Challenge at ICRA 2023 [Competitions]IEEE Robotics & Automation Magazine, 2023
- Leveraging Single-Goal Predictions to Improve the Efficiency of Multi-Goal Motion Planning with Dynamics2023
- Benchmarking Reinforcement Learning Techniques for Autonomous Navigation2023
- Autonomous Ground Navigation in Highly Constrained Spaces: Lessons Learned From the Benchmark Autonomous Robot Navigation Challenge at ICRA 2022 [Competitions]IEEE Robotics & Automation Magazine, 2022
- Machine Learning Methods for Local Motion Planning: A Study of End-to-End vs. Parameter Learning2021
- Benchmarking Metric Ground Navigation2020
- BaRC: Backward Reachability Curriculum for Robotic Reinforcement Learning2019