A Deep Learning Model of Perception in Color-Letter Synesthesia
Author:
Preprint
- 19 December 2017
- preprint
- Published by MDPI AG in Preprints.org
Abstract
Synesthesia is a psychological phenomenon where sensory signals become mixed. Input to one sensory modality produces an experience in a second, unstimulated modality. In “grapheme-color synesthesia”, viewed letters and numbers evoke mental imagery of colors. The study of this condition has implications for increasing our understanding of brain architecture and function, language, memory and semantics, and the nature of consciousness. In this work, we propose a novel application deep learning to model perception in grapheme-color synesthesia. Achromatic letter images, taken from database of handwritten characters, are used to induce synesthesia. Results show the model learns to accurately create a colored version of the inducing stimulus, according to a statistical distribution from experiments on a sample population of grapheme-color synesthetes. The spontaneous, creative mental imagery characteristic of the synesthetic perceptual experience is reproduced by the model. A model of synesthesia that generates testable predictions on brain activity and behavior is needed to complement large scale data collection efforts in neuroscience, especially when articulating simple descriptions of cause (stimulus) and effect (behavior). The research and modeling approach reported here begins to address this need.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Putting big data to good use in neuroscienceNature Neuroscience, 2014
- Magnetoencephalography reveals early activation of V4 in grapheme-color synesthesiaNeuroImage, 2010
- Learning multiple layers of representationTrends in Cognitive Sciences, 2007
- Effect of a Synesthete's Photisms on Name RecallCortex, 2006
- Neurocognitive Mechanisms of SynesthesiaNeuron, 2005
- A framework for consciousnessNature Neuroscience, 2003
- Anomalous perception in synaesthesia: A cognitive neuroscience perspectiveNature Reviews Neuroscience, 2002
- Psychophysical investigations into the neural basis of synaesthesiaProceedings Of The Royal Society B-Biological Sciences, 2001
- Mechanisms of synesthesia: cognitive and physiological constraintsTrends in Cognitive Sciences, 2001
- Five plus two equals yellowNature, 2000