Mapping Knowledge Representations to Concepts: A Review and New Perspectives
Preprint
Abstract
The success of neural networks builds to a large extent on their ability to create internal knowledge representations from real-world high-dimensional data, such as images, sound, or text. Approaches to extract and present these representations, in order to explain the neural network's decisions, is an active and multifaceted research field. To gain a deeper understanding of a central aspect of this field, we have performed a targeted review focusing on research that aims to associate internal representations with human understandable concepts. In doing this, we added a perspective on the existing research by using primarily deductive nomological explanations as a proposed taxonomy. We find this taxonomy and theories of causality, useful for understanding what can be expected, and not expected, from neural network explanations. The analysis additionally uncovers an ambiguity in the reviewed literature related to the goal of model explainability; is it understanding the ML model or, is it actionable explanations useful in the deployment domain?Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AIInformation Fusion, 2020
- The Costs of Connection: How Data Are Colonizing Human Life and Appropriating It for CapitalismSocial Forces, 2020
- Explainable Machine Learning for Scientific Insights and DiscoveriesIEEE Access, 2020
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based LocalizationInternational Journal of Computer Vision, 2019
- Explanation in artificial intelligence: Insights from the social sciencesArtificial Intelligence, 2019
- Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)IEEE Access, 2018
- Theory-Guided Data Science: A New Paradigm for Scientific Discovery from DataIEEE Transactions on Knowledge and Data Engineering, 2017
- Concept possession, experimental semantics, and hybrid theories of referencePhilosophical Psychology, 2012
- Carnap on concept determination: methodology for philosophy of scienceEuropean Journal for Philosophy of Science, 2011
- Mental Models and Causal Explanation: Judgements of Probable Cause and Explanatory RelevanceThinking & Reasoning, 1996