Global Concept-Based Interpretability for Graph Neural Networks via Neuron Analysis
Authors:
Preprint
Abstract
Graph neural networks (GNNs) are highly effective on a variety of graph-related tasks; however, they lack interpretability and transparency. Current explainability approaches are typically local and treat GNNs as black-boxes. They do not look inside the model, inhibiting human trust in the model and explanations. Motivated by the ability of neurons to detect high-level semantic concepts in vision models, we perform a novel analysis on the behaviour of individual GNN neurons to answer questions about GNN interpretability, and propose new metrics for evaluating the interpretability of GNN neurons. We propose a novel approach for producing global explanations for GNNs using neuron-level concepts to enable practitioners to have a high-level view of the model. Specifically, (i) to the best of our knowledge, this is the first work which shows that GNN neurons act as concept detectors and have strong alignment with concepts formulated as logical compositions of node degree and neighbourhood properties; (ii) we quantitatively assess the importance of detected concepts, and identify a trade-off between training duration and neuron-level interpretability; (iii) we demonstrate that our global explainability approach has advantages over the current state-of-the-art -- we can disentangle the explanation into individual interpretable concepts backed by logical descriptions, which reduces potential for bias and improves user-friendliness.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and ExplainabilityMachine Intelligence Research, 2024
- Explainability for experts: A design framework for making algorithms supporting expert decisions more explainableJournal of Responsible Technology, 2021
- A compact review of molecular property prediction with graph neural networksDrug Discovery Today: Technologies, 2020
- Explainable Machine Learning in Credit Risk ManagementComputational Economics, 2020
- XGNN: Towards Model-Level Explanations of Graph Neural Networks2020
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based LocalizationInternational Journal of Computer Vision, 2019
- A tutorial on kernel density estimation and recent advancesBiostatistics & Epidemiology, 2017
- Protein function prediction via graph kernelsBioinformatics, 2005
- Derivation and Validation of Toxicophores for Mutagenicity PredictionJournal of Medicinal Chemistry, 2004
- Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. Correlation with molecular orbital energies and hydrophobicityJournal of Medicinal Chemistry, 1991