Unexplainability and Incomprehensibility of AI
- 17 July 2020
- journal article
- research article
- Published by World Scientific Pub Co Pte Ltd in Journal of Artificial Intelligence and Consciousness
- Vol. 07 (02) , 277-291
Abstract
Explainability and comprehensibility of AI are important requirements for intelligent systems deployed in real-world domains. Users want and frequently need to understand how decisions impacting them are made. Similarly, it is important to understand how an intelligent system functions for safety and security reasons. In this paper, we describe two complementary impossibility results (Unexplainability and Incomprehensibility), essentially showing that advanced AIs would not be able to accurately explain some of their decisions and for the decisions they could explain people would not understand some of those explanations.Keywords
This publication has 56 references indexed in Scilit:
- Orthogonality-Based Disentanglement of Responsibilities for Ethical Intelligent Systems2019
- Activation AtlasDistill, 2019
- Automatic IQ Estimation Using Stylometric Methods2019
- Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)IEEE Access, 2018
- A Psychopathological Approach to Safety Engineering in AI and AGI2018
- Trends and Trajectories for Explainable, Accountable and Intelligible Systems2018
- Security Solutions for Intelligent and Complex Systems2017
- The Surveyability of Mathematical Proof: A Historical PerspectiveSynthese, 2006
- Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)Statistical Science, 2001
- Artificial liars: Why computers will (necessarily) deceive us and each otherEthics and Information Technology, 2000