Toward understanding compiler bugs in GCC and LLVM
Authors:
Open Access
- 18 July 2016
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 294-305
Abstract
Compilers are critical, widely-used complex software. Bugs in them have significant impact, and can cause serious damage when they silently miscompile a safety-critical application. An in-depth understanding of compiler bugs can help detect and fix them. To this end, we conduct the first empirical study on the characteristics of the bugs in two main-stream compilers, GCC and LLVM. Our study is significant in scale — it exhaustively examines about 50K bugs and 30K bug fix revisions over more than a decade’s span. This paper details our systematic study. Summary findings include: (1) In both compilers, C++ is the most buggy component, accounting for around 20% of the total bugs and twice as many as the second most buggy component; (2) the bug revealing test cases are typically small, with 80% having fewer than 45 lines of code; (3) most of the bug fixes touch a single source file with small modifications (43 lines for GCC and 38 for LLVM on average); (4) the average lifetime of GCC bugs is 200 days, and 111 days for LLVM; and (5) high priority tends to be assigned to optimizer bugs, most notably 30% of the bugs in GCC’s inter-procedural analysis component are labeled P1 (the highest priority). This study deepens our understanding of compiler bugs. For application developers, it shows that even mature production compilers still have many bugs, which may affect development. For researchers and compiler developers, it sheds light on interesting characteristics of compiler bugs, and highlights challenges and opportunities to more effectively test and debug compilers.Keywords
Funding Information
- NSF (1528133)
This publication has 32 references indexed in Scilit:
- Randomized stress-testing of link-time optimizers2015
- Provably correct peephole optimizations with aliveACM SIGPLAN Notices, 2015
- Provably correct peephole optimizations with alive2015
- Compiler validation via equivalence modulo inputsACM SIGPLAN Notices, 2014
- Testing Static Analyzers with Randomly Generated Programs2012
- An empirical study on configuration errors in commercial and open source systems2011
- How do fixes become bugs?2011
- Formal Verification of a C Compiler Front-EndLecture Notes in Computer Science, 2006
- Simplifying and isolating failure-inducing inputIEEE Transactions on Software Engineering, 2002
- Translation validation1998