DME-Driver: Integrating Human Decision Logic and 3D Scene Perception in Autonomous Driving
Top Cited Papers
Open Access
- 11 April 2025
- journal article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (3) , 3347-3355
Abstract
There are two crucial aspects of reliable autonomous driving systems: the reasoning behind decision-making and the precision of environmental perception. This paper introduces DME-Driver, a new autonomous driving system that enhances performance and robustness by fully leveraging the two crucial aspects. This system comprises two main models. The first, the Decision Maker, is responsible for providing logical driving instructions. The second, the Executor, receives these instructions and generates precise control signals for the vehicles. To ensure explainable and reliable driving decisions, we build the Decision-Maker based on a large vision language model. This model follows the logic employed by experienced human drivers and simulates making decisions in a safe and reasonable manner. On the other hand, the generation of accurate control signals relies on precise and detailed environmental perception, where 3D scene perception models excel. Therefore, a planning-oriented perception model is employed as the Executor. It translates the logical decisions made by the Decision-Maker into accurate control signals for the self-driving cars. To effectively train the proposed system, a new dataset named Human-driver Behavior and Decision-making (HBD) dataset has been collected. This dataset encompasses a diverse range of human driver behaviors and their underlying motivations. By leveraging this dataset, our system achieves high-precision planning accuracy through a logical thinking process.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agents2024
- Holistic Autonomous Driving Understanding by Bird'View Injected Multi-Modal Large Models2024
- Improved Baselines with Visual Instruction Tuning2024
- On the Road to Portability: Compressing End-to-End Motion Planner for Autonomous Driving2024
- LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs2024
- Driving Everywhere with Large Language Model Policy Adaptation2024
- VLP: Vision Language Planning for Autonomous Driving2024
- Semantic anomaly detection with large language modelsAutonomous Robots, 2023
- BEVFormer: Learning Bird’s-Eye-View Representation from Multi-camera Images via Spatiotemporal Transformers2022
- A survey of deep learning techniques for autonomous drivingJournal of Field Robotics, 2019