An Explanation Method for Models of Code
Open Access
- 16 October 2023
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Programming Languages
- Vol. 7 (OOPSLA2) , 801-827
Abstract
This paper introduces a novel method, called WheaCha, for explaining the predictions of code models. Similar to attribution methods, WheaCha seeks to identify input features that are responsible for a particular prediction that models make. On the other hand, it differs from attribution methods in crucial ways. Specifically, WheaCha separates an input program into "wheat" (i.e., defining features that are the reason for which models predict the label that they predict) and the rest "chaff" for any given prediction. We realize WheaCha in a tool, HuoYan, and use it to explain four prominent code models: code2vec, seq-GNN, GGNN, and CodeBERT. Results show that (1) HuoYan is efficient — taking on average under twenty seconds to compute wheat for an input program in an end-to-end fashion (i.e., including model prediction time); (2) the wheat that all models use to make predictions is predominantly comprised of simple syntactic or even lexical properties (i.e., identifier names); (3) neither the latest explainability methods for code models (i.e., SIVAND and CounterFactual Explanations) nor the most noteworthy attribution methods (i.e., Integrated Gradients and SHAP) can precisely capture wheat. Finally, we set out to demonstrate the usefulness of WheaCha, in particular, we assess if WheaCha’s explanations can help end users to identify defective code models (e.g., trained on mislabeled data or learned spurious correlations from biased data). We find that, with WheaCha, users achieve far higher accuracy in identifying faulty models than SIVAND, CounterFactual Explanations, Integrated Gradients and SHAP.Keywords
Funding Information
- National Natural Science Foundation of China (No. 62032010 and No. 62202220)
This publication has 23 references indexed in Scilit:
- Blended, precise semantic program embeddings2020
- Getafix: learning to fix bugs automaticallyProceedings of the ACM on Programming Languages, 2019
- SEQUENCER: Sequence-to-Sequence Learning for End-to-End Program RepairIEEE Transactions on Software Engineering, 2019
- Deep learning predicts hip fracture using confounding patient and healthcare variablesnpj Digital Medicine, 2019
- code2vec: learning distributed representations of codeProceedings of the ACM on Programming Languages, 2019
- DeepBugs: a learning approach to name-based bug detectionProceedings of the ACM on Programming Languages, 2018
- "Why Should I Trust You?"2016
- On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance PropagationPLOS ONE, 2015
- Simplifying and isolating failure-inducing inputIEEE Transactions on Software Engineering, 2002
- Yesterday, my program worked. Today, it does not. Why?ACM SIGSOFT Software Engineering Notes, 1999