FuzzyFlow: Leveraging Dataflow To Find and Squash Program Optimization Bugs
Preprint
Abstract
The current hardware landscape and application scale is driving performance engineers towards writing bespoke optimizations. Verifying such optimizations, and generating minimal failing cases, is important for robustness in the face of changing program conditions, such as inputs and sizes. However, isolation of minimal test-cases from existing applications and generating new configurations are often difficult due to side effects on the system state, mostly related to dataflow. This paper introduces FuzzyFlow: a fault localization and test case extraction framework designed to test program optimizations. We leverage dataflow program representations to capture a fully reproducible system state and area-of-effect for optimizations to enable fast checking for semantic equivalence. To reduce testing time, we design an algorithm for minimizing test inputs, trading off memory for recomputation. We demonstrate FuzzyFlow on example use cases in real-world applications where the approach provides up to 528 times faster optimization testing and debugging compared to traditional approaches.Keywords
All Related Versions
This publication has 53 references indexed in Scilit:
- Taco: A tool to generate tensor algebra kernels2017
- Specifying and executing optimizations for generalized control flow graphsScience of Computer Programming, 2016
- Practical testing of a C99 compiler using output comparisonSoftware: Practice and Experience, 2007
- Translation and Run-Time Validation of Loop TransformationsFormal Methods in System Design, 2005
- LLVM: A compilation framework for lifelong program analysis & transformation2004
- Simplifying and isolating failure-inducing inputIEEE Transactions on Software Engineering, 2002
- VOCElectronic Notes in Theoretical Computer Science, 2002
- The Code Validation Tool (CVT)International Journal on Software Tools for Technology Transfer, 1998
- Translation validation1998
- Compiler test case generation methods: a survey and assessmentInformation and Software Technology, 1997