"I know it when I see it". Visualization and Intuitive Interpretability
Author:
Preprint
Abstract
Most research on the interpretability of machine learning systems focuses on the development of a more rigorous notion of interpretability. I suggest that a better understanding of the deficiencies of the intuitive notion of interpretability is needed as well. I show that visualization enables but also impedes intuitive interpretability, as it presupposes two levels of technical pre-interpretation: dimensionality reduction and regularization. Furthermore, I argue that the use of positive concepts to emulate the distributed semantic structure of machine learning models introduces a significant human bias into the model. As a consequence, I suggest that, if intuitive interpretability is needed, singular representations of internal model states should be avoided.Keywords
All Related Versions
This publication has 5 references indexed in Scilit:
- One Pixel Attack for Fooling Deep Neural NetworksIEEE Transactions on Evolutionary Computation, 2019
- Adversarial Examples in the Physical World2018
- Visualizing Deep Convolutional Neural Networks Using Natural Pre-imagesInternational Journal of Computer Vision, 2016
- The Range of InterpretationWorld Literature Today, 2001
- Extensions of Lipschitz mappings into a Hilbert space1984