Mental image reconstruction from human brain activity
Preprint
Abstract
Visual images perceived by humans can be reconstructed from their brain activity. However, the visualization (externalization) of mental imagery remains a challenge. In this study, we demonstrated that the visual image reconstruction method proposed in the seminal study by Shen et al. (2019) heavily relied on low-level visual information decoded from the brain and could not efficiently utilize semantic information that would be recruited during mental imagery. To address this limitation, we extended the previous method to a Bayesian estimation framework and introduced the assistance of semantic information into it. Our proposed framework successfully reconstructed both seen (i.e., directly captured by the human eye) and imagined images from the brain activity. These results suggest that our framework would provide a technology for directly investigating the subjective contents of the brain.Keywords
All Related Versions
- Published version: Neural Networks, 170, 349.
This publication has 37 references indexed in Scilit:
- Decoding Working Memory of Stimulus Contrast in Early Visual CortexJournal of Neuroscience, 2013
- Modular Encoding and Decoding Models Derived from Bayesian Canonical Correlation AnalysisNeural Computation, 2013
- Disentangling visual imagery and perception of real-world objectsNeuroImage, 2012
- Reconstructing Visual Experiences from Brain Activity Evoked by Natural MoviesCurrent Biology, 2011
- Reading the mind's eye: Decoding category information during mental imageryNeuroImage, 2010
- Bayesian Reconstruction of Natural Images from Human Brain ActivityNeuron, 2009
- I can see what you seeNature Neuroscience, 2009
- Decoding reveals the contents of visual working memory in early visual areasNature, 2009
- Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image DecodersNeuron, 2008
- Identifying natural images from human brain activityNature, 2008