PRIX: Learning to Plan From Raw Pixels for End-to-End Autonomous Driving
Open Access
- 30 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (5) , 6400-6407
Abstract
While end-to-end autonomous driving models show promising results, their practical deployment is often hindered by large model sizes, a reliance on expensive LiDAR sensors and computationally intensive BEV feature representations. This limits their scalability, especially for mass-market vehicles equipped only with cameras. To address these challenges, we propose PRIX (Plan from Raw pIXels). Our novel and efficient end-to-end driving architecture operates using only camera data, without explicit BEV representation and forgoing the need for LiDAR. PRIX leverages a visual feature extractor coupled with a generative planning head to predict safe trajectories from raw pixel inputs directly. A core component of our architecture is the Context-aware Recalibration Transformer (CaRT), a novel module designed to effectively enhance multi-level visual features for more robust planning. PRIX achieves SOTA performance on the NavSim-v2 and nuScenes datasets. On NavSim-v1, it also outperforms the majority of multimodal planners and other camera-only approaches on most metrics. Critically, PRIX is significantly more efficient on NavSim-v1, boasting faster inference speeds and a smaller model size. This combination of performance and efficiency makes it a practical solution for real-world deployment.Keywords
Funding Information
- Wallenberg Ai, Autonomous Systems and Software Program
This publication has 27 references indexed in Scilit:
- GoalFlow: Goal-Driven Flow Matching for Multimodal Trajectories Generation in End-to-End Autonomous Driving2025
- Fully Sparse 3D Occupancy Prediction2024
- PPAD: Iterative Interactions of Prediction and Planning for End-to-End Autonomous Driving2024
- PARA-Drive: Parallelized Architecture for Real-Time Autonomous Driving2024
- NeuRAD: Neural Rendering for Autonomous Driving2024
- Planning-oriented Autonomous Driving2023
- ST-P3: End-to-End Vision-Based Autonomous Driving via Spatial-Temporal Feature Learning2022
- Lift, Splat, Shoot: Encoding Images from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D2020
- nuScenes: A Multimodal Dataset for Autonomous Driving2020
- Squeeze-and-Excitation Networks2018