A Hierarchical NMPC and TD3-Based Framework for Seamless Cruise-to-Park Automated Valet Parking
Open Access
Abstract
Automated valet parking requires reliable long-range slot searching and precise low-speed docking in confined structured lots. This paper proposes a hierarchical cruise-to-park framework that combines nonlinear model predictive control (NMPC) for predefined-route cruising with a Twin Delayed Deep Deterministic Policy Gradient (TD3) agent for terminal parking. The system is implemented in a structured Simulink environment with Unreal Engine-based geometry-aware sensing modules. During cruising, a camera-based module detects available slots and triggers the transition to parking. The NMPC uses a custom cost function to improve tracking on curved approaches, while the TD3 policy uses LiDAR feedback and reward shaping with an explicit time penalty to encourage efficient, stable docking. Simulation results demonstrate smooth phase transition, accurate cruising, and effective terminal parking in the training slot. Validation on six previously unseen target slots within the same parking-lot environment shows encouraging intra-lot target-slot transferability without retraining. Additional PPO and SAC comparisons and a time-penalty ablation further evaluate the relative learning performance and the effect of reward design, supporting the proposed architecture as a practical baseline for integrated cruise-to-park automated valet parking studies.Keywords
All Related Versions
This publication has 43 references indexed in Scilit:
- A reinforcement learning‐based reverse‐parking system for autonomous vehiclesIET Intelligent Transport Systems, 2025
- Model Predictive Control and Reinforcement Learning: A Unified Framework Based on Dynamic ProgrammingIFAC-PapersOnLine, 2024
- Toward autonomous vehicles: A survey on cooperative vehicle-infrastructure systemiScience, 2024
- Deep reinforcement learning-based drift parking control of automated vehiclesScience China Technological Sciences, 2023
- Reward (Mis)design for autonomous drivingArtificial Intelligence, 2022
- From smart parking towards autonomous valet parking: A survey, challenges and future WorksJournal of Network and Computer Applications, 2021
- A novel path planning methodology for automated valet parking based on directional graph search and geometry curveRobotics and Autonomous Systems, 2020
- Reinforcement Learning-Based Motion Planning for Automatic Parking SystemIEEE Access, 2020
- Trajectory Planning of Autonomous Vehicles Based on Parameterized Control Optimization in Dynamic on-Road EnvironmentsJournal of Intelligent & Robotic Systems, 2020
- Constrained model predictive control: Stability and optimalityAutomatica, 2000