ROAD : R esponsibility- O riented Reward Design for Reinforcement Learning in A utonomous D riving
- 1 June 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (6) , 6552-6559
Abstract
Reinforcementlearning (RL) in autonomous driving employs a trial-and-error mechanism, enhancing robustness in unpredictable environments. However, crafting effective reward functions remains challenging, as conventional approaches rely heavily on manual design and demonstrate limited efficacy in complex scenarios. To address this issue, this study introduces a responsibility-oriented reward function that explicitly incorporates traffic regulations into the RL framework. Specifically, we first use a vision language model (VLM) to determine the liability in traffic collisions and propose reward signals. To mitigate VLM hallucination and ensure regulatory grounding, we propose a Traffic Regulation Knowledge Graph (TRKG) that structures unstructured traffic laws into a queryable ontology of driving scenarios and liability standards. This mechanism retrieves the precise regulatory context required to quantify accident responsibility (primary/shared/secondary), which is then used to modulate the crash penalty in the reward function. Building on these liability-informed signals, we construct a reward function and employ it to train the agent, thereby encouraging behavior that more faithfully adheres to traffic regulations. Experimental validations show that our approach achieves task success rates of 73.2% (intersection) and 54.0% (roundabout). Compared with the original policy, the success rate improves by +8.2 pp/+11.2 pp, while the ego vehicle's primary-liability share decreases by 13.5 pp/5.7 pp. The code for this work is available at https://github.com/Songan-Lab/ROAD.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (52402504)
This publication has 25 references indexed in Scilit:
- VLM-RL: A unified vision language models and reinforcement learning framework for safe autonomous drivingTransportation Research Part C: Emerging Technologies, 2025
- PE-RLHF: Reinforcement Learning with Human Feedback and physics knowledge for safe and trustworthy autonomous drivingTransportation Research Part C: Emerging Technologies, 2025
- A large language model-driven reward design framework via dynamic feedback for reinforcement learningKnowledge-Based Systems, 2025
- LingoQA: Visual Question Answering for Autonomous Driving2024
- Modelling Competitive Behaviors in Autonomous Driving Under Generative World Model2024
- Intermediate Tasks Enhanced End-to-End Autonomous Driving with Uncertainty Estimation2024
- Design of Reward Function on Reinforcement Learning for Automated DrivingIFAC-PapersOnLine, 2023
- Reveal: Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memory2023
- Reward (Mis)design for autonomous drivingArtificial Intelligence, 2022
- Class-Balanced Loss Based on Effective Number of Samples2019