Assessing LLM code generation quality through path planning tasks
Preprint
Abstract
As LLM-generated code grows in popularity, more evaluation is needed to assess the risks of using such tools, especially for safety-critical applications such as path planning. Existing coding benchmarks are insufficient as they do not reflect the context and complexity of safety-critical applications. To this end, we assessed six LLMs' abilities to generate the code for three different path-planning algorithms and tested them on three maps of various difficulties. Our results suggest that LLM-generated code presents serious hazards for path planning applications and should not be applied in safety-critical contexts without rigorous testing.Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- A Survey on Large Language Models for Code GenerationACM Transactions on Software Engineering and Methodology, 2026
- Bugs in large language models generated code: an empirical studyEmpirical Software Engineering, 2025
- A Path-Planning Approach Based on Potential and Dynamic Q-Learning for Mobile Robots in Unknown EnvironmentComputational Intelligence and Neuroscience, 2022
- Analysis of the change in bugginess and adaptiveness of python software systemsMultimedia Tools and Applications, 2022
- Voronoi diagrams—a survey of a fundamental geometric data structureACM Computing Surveys, 1991
- An Algorithm for Finding Best Matches in Logarithmic Expected TimeACM Transactions on Mathematical Software, 1977
- A note on two problems in connexion with graphsNumerische Mathematik, 1959