Learning to Control Autonomous Fleets from Observation via Offline Reinforcement Learning
Preprint
Abstract
Autonomous Mobility-on-Demand (AMoD) systems are an evolving mode of transportation in which a centrally coordinated fleet of self-driving vehicles dynamically serves travel requests. The control of these systems is typically formulated as a large network optimization problem, and reinforcement learning (RL) has recently emerged as a promising approach to solve the open challenges in this space. Recent centralized RL approaches focus on learning from online data, ignoring the per-sample-cost of interactions within real-world transportation systems. To address these limitations, we propose to formalize the control of AMoD systems through the lens of offline reinforcement learning and learn effective control strategies using solely offline data, which is readily available to current mobility operators. We further investigate design decisions and provide empirical evidence based on data from real-world mobility systems showing how offline learning allows to recover AMoD control policies that (i) exhibit performance on par with online methods, (ii) allow for sample-efficient online fine-tuning and (iii) eliminate the need for complex simulation environments. Crucially, this paper demonstrates that offline RL is a promising paradigm for the application of RL-based solutions within economically-critical systems, such as mobility systems.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open ProblemsIEEE Transactions on Neural Networks and Learning Systems, 2023
- Offline Reinforcement Learning for Autonomous Driving with Real World Driving Data2022
- Reinforcement learning for ridesharing: An extended surveyTransportation Research Part C: Emerging Technologies, 2022
- Multi-objective vehicle rebalancing for ridehailing system using a reinforcement learning approachJournal of Management Science and Engineering, 2022
- Real-world ride-hailing vehicle repositioning using deep reinforcement learningTransportation Research Part C: Emerging Technologies, 2021
- Scalable Deep Reinforcement Learning for Ride-HailingIEEE Control Systems Letters, 2020
- Dispatch of autonomous vehicles for taxi services: A deep reinforcement learning approachTransportation Research Part C: Emerging Technologies, 2020
- Dynamic autonomous vehicle fleet operations: Optimization-based strategies to assign AVs to immediate traveler demand requestsTransportation Research Part C: Emerging Technologies, 2018
- A general framework for modeling shared autonomous vehicles with dynamic network-loading and dynamic ride-sharing applicationComputers, Environment and Urban Systems, 2017
- CRAWDAD dataset roma/taxi (v. 2014-07-17)