Graph Convolutional Network Based Multi-Objective Meta-Deep Q-Learning for Eco-Routing
- 16 January 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 25 (7) , 7323-7338
Abstract
Route selection can greatly affect vehicle fuel consumption and emissions. Finding the most fuel/energy-efficient route is known as the eco-routing problem. Existing eco-routing solutions do not effectively consider the critical traffic signal information and rely on fuel consumption models that may not be sufficiently accurate. To address the eco-routing problem in a signalized traffic network, this paper proposes a graph convolutional network based multi-objective meta-deep Q-learning (GM $^{\bm{2}}$ DQL) method. The problem is formulated as dynamic multi-objective Markov decision processes (MOMDP) and is tackled through deep reinforcement learning and meta-learning. We identify that graph convolutional network (GCN) is an efficient and suitable feature representation for a signalized traffic network. GM $^{\bm{2}}$ DQL can explore the optimal routes with respect to drivers’ different preferences on saving fuel and travel time. Through GM $^{\bm{2}}$ DQL, the agent is trained under a series of learning environments that are characterized by historical vehicle trajectories, fuel consumption data, and traffic signal data in the remote data center. The vehicle requesting eco-routing service can download the model that represents the action value function of the historical dynamic driving conditions. The model in the vehicle can quickly adapt to the most recent driving condition through online one-shot learning and predict the optimal eco-routes for the subsequent unseen driving conditions of the signalized traffic network. Extensive proof-of-concept experiments validate that GM $^{\bm{2}}$ DQL can effectively discover optimal eco-routes. It saves up to 71% travel time and 62% fuel, compared to the conventional shortest-path routing strategy that is widely used in navigation systems.Keywords
Funding Information
- National Science Foundation (CNS-0644056)
- UML Internal Seed Grant
This publication has 36 references indexed in Scilit:
- Electric vehicle routing problem with machine learning for energy predictionTransportation Research Part B: Methodological, 2021
- Greenhouse gas emission prediction on road network using deep sequence learningTransportation Research Part D: Transport and Environment, 2020
- Vector-based navigation using grid-like representations in artificial agentsNature, 2018
- Application of the support vector machine and heuristic k-shortest path algorithm to determine the most eco-friendly path with a travel time constraintTransportation Research Part D: Transport and Environment, 2017
- A review of vehicle fuel consumption models to evaluate eco-driving and eco-routingTransportation Research Part D: Transport and Environment, 2016
- Stochastic Eco-routing in a Signalized Traffic NetworkTransportation Research Procedia, 2015
- A review of recent research on green road freight transportationEuropean Journal of Operational Research, 2014
- An eco-routing model considering microscopic vehicle operating conditionsTransportation Research Part B: Methodological, 2013
- Parallel reinforcement learning for weighted multi-criteria model with adaptive marginCognitive Neurodynamics, 2008
- Optimizing route choice for lowest fuel consumption – Potential effects of a new driver support toolTransportation Research Part C: Emerging Technologies, 2006