ReLU Code Space: A Basis for Rating Network Quality Besides Accuracy
Preprint
Abstract
We propose a new metric space of ReLU activation codes equipped with a truncated Hamming distance which establishes an isometry between its elements and polyhedral bodies in the input space which have recently been shown to be strongly related to safety, robustness, and confidence. This isometry allows the efficient computation of adjacency relations between the polyhedral bodies. Experiments on MNIST and CIFAR-10 indicate that information besides accuracy might be stored in the code space.Keywords
All Related Versions
This publication has 5 references indexed in Scilit:
- Analysis of the Generalization Error: Empirical Risk Minimization over Deep Artificial Neural Networks Overcomes the Curse of Dimensionality in the Numerical Approximation of Black--Scholes Partial Differential EquationsSIAM Journal on Mathematics of Data Science, 2020
- Scaling up the Randomized Gradient-Free Adversarial Attack Reveals Overestimation of Robustness Using Established AttacksInternational Journal of Computer Vision, 2019
- Why ReLU Networks Yield High-Confidence Predictions Far Away From the Training Data and How to Mitigate the Problem2019
- An Introduction to Convex Polytopes1983
- Representations of polytopes and polyhedral setsGeometriae Dedicata, 1973