Characterization of deep neural network features by decodability from human brain activity
Open Access
- 12 February 2019
- journal article
- research article
- Published by Springer Nature in Scientific Data
- Vol. 6 (1) , 190012
Abstract
Achievements of near human-level performance in object recognition by deep neural networks (DNNs) have triggered a flood of comparative studies between the brain and DNNs. Using a DNN as a proxy for hierarchical visual representations, our recent study found that human brain activity patterns measured by functional magnetic resonance imaging (fMRI) can be decoded (translated) into DNN feature values given the same inputs. However, not all DNN features are equally decoded, indicating a gap between the DNN and human vision. Here, we present a dataset derived from DNN feature decoding analyses, which includes fMRI signals of five human subjects during image viewing, decoded feature values of DNNs (AlexNet and VGG19), and decoding accuracies of individual DNN features with their rankings. The decoding accuracies of individual features were highly correlated between subjects, suggesting the systematic differences between the brain and DNNs. We hope the present dataset will contribute to revealing the gap between the brain and DNNs and provide an opportunity to make use of the decoded features for further applications.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- Visualizing Deep Convolutional Neural Networks Using Natural Pre-imagesInternational Journal of Computer Vision, 2016
- Using goal-driven deep learning models to understand sensory cortexNature Neuroscience, 2016
- Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral StreamThe Journal of Neuroscience, 2015
- Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object RecognitionPLOS Computational Biology, 2014
- Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical RepresentationPLOS Computational Biology, 2014
- Performance-optimized hierarchical models predict neural responses in higher visual cortexProceedings of the National Academy of Sciences of the United States of America, 2014
- Online Model Selection Based on the Variational BayesNeural Computation, 2001
- A cortical representation of the local visual environmentNature, 1998
- Borders of Multiple Visual Areas in Humans Revealed by Functional Magnetic Resonance ImagingScience, 1995
- fMRI of human visual cortexNature, 1994