Reconstructing imagined letters from early visual cortex reveals tight topographic correspondence between visual mental imagery and perception
Open Access
- 14 January 2019
- journal article
- research article
- Published by Springer Nature in Brain Structure and Function
- Vol. 224 (3) , 1167-1183
Abstract
Visual mental imagery is the quasi-perceptual experience of “seeing in the mind’s eye”. While a tight correspondence between imagery and perception in terms of subjective experience is well established, their correspondence in terms of neural representations remains insufficiently understood. In the present study, we exploit the high spatial resolution of functional magnetic resonance imaging (fMRI) at 7T, the retinotopic organization of early visual cortex, and machine-learning techniques to investigate whether visual imagery of letter shapes preserves the topographic organization of perceived shapes. Sub-millimeter resolution fMRI images were obtained from early visual cortex in six subjects performing visual imagery of four different letter shapes. Predictions of imagery voxel activation patterns based on a population receptive field-encoding model and physical letter stimuli provided first evidence in favor of detailed topographic organization. Subsequent visual field reconstructions of imagery data based on the inversion of the encoding model further showed that visual imagery preserves the geometric profile of letter shapes. These results open new avenues for decoding, as we show that a denoising autoencoder can be used to pretrain a classifier purely based on perceptual data before fine-tuning it on imagery data. Finally, we show that the autoencoder can project imagery-related voxel activations onto their perceptual counterpart allowing for visually recognizable reconstructions even at the single-trial level. The latter may eventually be utilized for the development of content-based BCI letter-speller systems.Keywords
All Related Versions
Funding Information
- European Research Council (269853)
- Horizon 2020 (7202070)
- European Research Council (779860)
This publication has 48 references indexed in Scilit:
- Disentangling visual imagery and perception of real-world objectsNeuroImage, 2012
- Metamers of the ventral streamNature Neuroscience, 2011
- Multiband multislice GE‐EPI at 7 tesla, with 16‐fold acceleration using partial parallel imaging with application to high spatial and temporal whole‐brain fMRIMagnetic Resonance in Medicine, 2010
- Reading the mind's eye: Decoding category information during mental imageryNeuroImage, 2010
- MP2RAGE, a self bias-field corrected sequence for improved segmentation and T1-mapping at high fieldNeuroImage, 2010
- Decoding reveals the contents of visual working memory in early visual areasNature, 2009
- Population receptive field estimates in human visual cortexNeuroImage, 2008
- PsychoPy—Psychophysics software in PythonJournal of Neuroscience Methods, 2007
- Inverse retinotopy: Inferring the visual content of images from brain activation patternsNeuroImage, 2006
- Are Theories of Imagery Theories of Imagination? An Active Perception Approach to Conscious Mental ContentCognitive Science, 1999