Brain hierarchy score: Which deep neural networks are hierarchically brain-like?
Preprint
Abstract
Summary: Achievement of human-level image recognition by deep neural networks (DNNs) has spurred interest in whether and how DNNs are brain-like. Both DNNs and the visual cortex perform hierarchical processing, and correspondence has been shown between hierarchical visual areas and DNN layers in representing visual features. Here, we propose the brain hierarchy (BH) score as a metric to quantify the degree of hierarchical correspondence based on the decoding of individual DNN unit activations from human brain activity. We find that BH scores for 29 pretrained DNNs with varying architectures are negatively correlated with image recognition performance, indicating that recently developed high-performance DNNs are not necessarily brain-like. Experimental manipulations of DNN models suggest that relatively simple feedforward architecture with broad spatial integration is critical to brain-like hierarchy. Our method provides new ways for designing DNNs and understanding the brain in consideration of their representational homology.Keywords
All Related Versions
- Published version: iScience, 24 (9), 103013.
This publication has 39 references indexed in Scilit:
- Performance-optimized hierarchical models predict neural responses in higher visual cortexProceedings of the National Academy of Sciences of the United States of America, 2014
- Spatial coding and invariance in object-selective cortexCortex, 2011
- Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categoriesComputer Vision and Image Understanding, 2007
- Decoding the visual and subjective contents of the human brainNature Neuroscience, 2005
- Responses of human visual cortex to uniform surfacesProceedings of the National Academy of Sciences, 2004
- The Small World of the Cerebral CortexNeuroinformatics, 2004
- Borders of Multiple Visual Areas in Humans Revealed by Functional Magnetic Resonance ImagingScience, 1995
- fMRI of human visual cortexNature, 1994
- Preface: Cerebral Cortex Has Come of AgeCerebral Cortex, 1991
- SOME QUESTIONS OF DISTRIBUTION IN THE THEORY OF RANK CORRELATIONBiometrika, 1951