Improving Compiler Bug Isolation by Leveraging Large Language Models
Preprint
Abstract
Compilers play a foundational role in building reliable software systems, and bugs within them can lead to catastrophic consequences. The compilation process typically involves hundreds of files, making traditional automated bug isolation techniques inapplicable due to scalability or effectiveness issues. Current mainstream compiler bug localization techniques have limitations in test program mutation and resource consumption. Inspired by the recent advances of pre-trained Large Language Models (LLMs), we propose an innovative approach named AutoCBI, which (1) uses LLMs to summarize compiler file functions and (2) employs specialized prompts to guide LLM in reordering suspicious file rankings. This approach leverages four types of information: the failing test program, source file function summaries, lists of suspicious files identified through analyzing test coverage, as well as compilation configurations with related output messages, resulting in a refined ranking of suspicious files. Our evaluation of AutoCBI against state-of-the-art approaches (DiWi, RecBi and FuseFL) on 120 real-world bugs from the widely-used GCC and LLVM compilers demonstrates its effectiveness. Specifically, AutoCBI isolates 66.67%/69.23%, 300%/340%, and 100%/57.14% more bugs than RecBi, DiWi, and FuseFL, respectively, in the Top-1 ranked results for GCC/LLVM. Additionally, the ablation study underscores the significance of each component in our approach.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Isolating Compiler Bugs by Generating Effective Witness Programs With Large Language ModelsIEEE Transactions on Software Engineering, 2024
- LocSeq: Automated Localization for Compiler Optimization Sequence Bugs of LLVMIEEE Transactions on Reliability, 2022
- A Survey of Compiler TestingACM Computing Surveys, 2020
- Historical Spectrum Based Fault LocalizationIEEE Transactions on Software Engineering, 2019
- A manual inspection of Defects4J bugs and its implications for automatic program repairScience China Information Sciences, 2019
- Transforming programs and tests in tandem for fault localizationProceedings of the ACM on Programming Languages, 2017
- Practitioners' expectations on automated fault localization2016
- The DStar Method for Effective Software Fault LocalizationIEEE Transactions on Reliability, 2013
- Injecting mechanical faults to localize developer faults for evolving softwareACM SIGPLAN Notices, 2013
- Where should the bugs be fixed? More accurate information retrieval-based bug localization based on bug reports2012