David A. Tovar
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
David A. Tovar has a total of 64 co-authors. Here are the top co-authors by number of shared publications:
- Mark T. Wallace
Vanderbilt Brain Institute, Vanderbilt University, Nashville, USA
6 shared publications - Micah M. Murray
Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
6 shared publications - Chrysa Retsa
The Sense Innovation and Research Center, Lausanne and Sion, Switzerland
2 shared publications - David J. Lewkowicz
Child Study Center, Yale School of Medicine, New Haven, CT, USA
2 shared publications - Katia Steinfeld
The Radiology Department, Lausanne University Hospital and University of Lausanne, 1011 Lausanne, Switzerland
2 shared publications Mark T. Wallace
Neuroscience Program, Vanderbilt University, Nashville, TN, USA
2 shared publications
Latest Publications
- 12 September 2026
- Published by Elsevier in Developmental Review
- Vol. 82
The role of multisensory integration in child development: Perception, cognition, and action
- 12 September 2026
- Published by Elsevier in Developmental Review
- Vol. 82
The role of multisensory integration in child development: Perception, cognition, and action
- Open Access
- 1 February 2026
- Published by Center for Open Science in PsyArXiv
Fundamentals of Topographic Analyses of Event-Related Potentials
Electroencephalography (EEG) marked its centennial in 2024 and remains a major workhorse in clinical, basic, and applied research alike. This is partially attributable to EEG’s accessibility and cost-efficiency as well as the fact that EEG non-invasively measures the electric fields of the brain. Innovations in EEG over the past century have largely been either in terms of improving data acquisition or data analysis. On the one hand, the hardware for EEG recording allows for more electrodes from more efficient electronics and well as more user-friendly hardware. On the other hand, advancements in signal processing allow researchers to use EEG to provide both temporal and spatial information about brain (dys)function. However, that EEG is so widespread also makes the method susceptible to practices that may hinder scientific advancement by adding confusion rather than clarity in the acquisition, analysis, and interpretation of EEG data. This chapter offers readers an accessible overview of these considerations as well as a guide to these analyses that provide reference-independent measurements of response strength and response topography and therefore circumvent statistical and interpretational caveats of canonical ERP analysis methods. The analyses overviewed here can, in most instances, be extended to the analysis of spontaneous EEG, as well as spectrally-deconvolved signals both when recorded at the scalp and intracranially in humans and animal models alike. The versatility and accessibility of EEG, in combination with advances in signal processing, allow for this ‘old dog’ of human neuroscience to still deliver new tricks and innovations.
- 1 February 2026
- Published by Center for Open Science in PsyArXiv
Open AccessFundamentals of Topographic Analyses of Event-Related Potentials
Electroencephalography (EEG) marked its centennial in 2024 and remains a major workhorse in clinical, basic, and applied research alike. This is partially attributable to EEG’s accessibility and cost-efficiency as well as the fact that EEG non-invasively measures the electric fields of the brain. Innovations in EEG over the past century have largely been either in terms of improving data acquisition or data analysis. On the one hand, the hardware for EEG recording allows for more electrodes from more efficient electronics and well as more user-friendly hardware. On the other hand, advancements in signal processing allow researchers to use EEG to provide both temporal and spatial information about brain (dys)function. However, that EEG is so widespread also makes the method susceptible to practices that may hinder scientific advancement by adding confusion rather than clarity in the acquisition, analysis, and interpretation of EEG data. This chapter offers readers an accessible overview of these considerations as well as a guide to these analyses that provide reference-independent measurements of response strength and response topography and therefore circumvent statistical and interpretational caveats of canonical ERP analysis methods. The analyses overviewed here can, in most instances, be extended to the analysis of spontaneous EEG, as well as spectrally-deconvolved signals both when recorded at the scalp and intracranially in humans and animal models alike. The versatility and accessibility of EEG, in combination with advances in signal processing, allow for this ‘old dog’ of human neuroscience to still deliver new tricks and innovations.
- 1 August 2025
- Published by Walter de Gruyter GmbH in Multisensory Research
- Vol. -1 (aop) , 1-20
CART: The Comprehensive Analysis of Reaction Times — GUI for Multisensory Processes and Race Models
Multisensory integration (MSI) is a core neurobehavioral operation that enhances our ability to perceive, decide, and act by combining information from different sensory modalities. This integrative capability is essential for efficiently navigating complex environments and responding to their multisensory nature. One of the powerful behavioral benefits of MSI is in speeding responses. To evaluate this speeding, traditional research in MSI often relies on so-called race models, which predict reaction times (RTs) based on the assumption that information from the different sensory modalities is initially processed independently. When observed RTs are faster than those predicted by these models, it indicates the presence of true convergence and integration of multisensory information prior to the initiation of the motor response. Despite the strong applicability of race models in MSI research, analysis of multisensory RT data often poses challenges for researchers, particularly in managing, interpreting and modeling large datasets or a collection of datasets. To surmount these challenges, we developed a user-friendly graphical user interface (GUI) packaged into a freely available software application that is compatible with both Windows and Mac and that requires no programming expertise. This tool simplifies the processes of data loading, filtering, and statistical analysis. It allows the calculation and visualization of RTs across different sensory modalities, the performance of robust statistical tests, and the testing of race model violations. By integrating these capabilities into a single platform, the CART-GUI facilitates MSI analyses and makes it accessible to a wider range of users, from novice researchers to experts in the field. The GUI’s user-friendly design and advanced analytical features will allow for valuable insights into the mechanisms underlying MSI and contribute to the advancement of research in this domain.
- 1 August 2025
- Published by Walter de Gruyter GmbH in Multisensory Research
- Vol. -1 (aop) , 1-20
CART: The Comprehensive Analysis of Reaction Times — GUI for Multisensory Processes and Race Models
Multisensory integration (MSI) is a core neurobehavioral operation that enhances our ability to perceive, decide, and act by combining information from different sensory modalities. This integrative capability is essential for efficiently navigating complex environments and responding to their multisensory nature. One of the powerful behavioral benefits of MSI is in speeding responses. To evaluate this speeding, traditional research in MSI often relies on so-called race models, which predict reaction times (RTs) based on the assumption that information from the different sensory modalities is initially processed independently. When observed RTs are faster than those predicted by these models, it indicates the presence of true convergence and integration of multisensory information prior to the initiation of the motor response. Despite the strong applicability of race models in MSI research, analysis of multisensory RT data often poses challenges for researchers, particularly in managing, interpreting and modeling large datasets or a collection of datasets. To surmount these challenges, we developed a user-friendly graphical user interface (GUI) packaged into a freely available software application that is compatible with both Windows and Mac and that requires no programming expertise. This tool simplifies the processes of data loading, filtering, and statistical analysis. It allows the calculation and visualization of RTs across different sensory modalities, the performance of robust statistical tests, and the testing of race model violations. By integrating these capabilities into a single platform, the CART-GUI facilitates MSI analyses and makes it accessible to a wider range of users, from novice researchers to experts in the field. The GUI’s user-friendly design and advanced analytical features will allow for valuable insights into the mechanisms underlying MSI and contribute to the advancement of research in this domain.
- Open Access
Beginning with You: Perceptual-Initialization Improves Vision-Language Representation and Alignment
We introduce Perceptual-Initialization (PI), a paradigm shift in visual representation learning that incorporates human perceptual structure during the initialization phase rather than as a downstream fine-tuning step. By integrating human-derived triplet embeddings from the NIGHTS dataset to initialize a CLIP vision encoder, followed by self-supervised learning on YFCC15M, our approach demonstrates significant zero-shot performance improvements, without any task-specific fine-tuning, across 29 zero shot classification and 2 retrieval benchmarks. On ImageNet-1K, zero-shot gains emerge after approximately 15 epochs of pretraining. Benefits are observed across datasets of various scales, with improvements manifesting at different stages of the pretraining process depending on dataset characteristics. Our approach consistently enhances zero-shot top-1 accuracy, top-5 accuracy, and retrieval recall (e.g., R@1, R@5) across these diverse evaluation tasks, without requiring any adaptation to target domains. These findings challenge the conventional wisdom of using human-perceptual data primarily for fine-tuning and demonstrate that embedding human perceptual structure during early representation learning yields more capable and vision-language aligned systems that generalize immediately to unseen tasks. Our work shows that "beginning with you", starting with human perception, provides a stronger foundation for general-purpose vision-language intelligence.
Open AccessBeginning with You: Perceptual-Initialization Improves Vision-Language Representation and Alignment
We introduce Perceptual-Initialization (PI), a paradigm shift in visual representation learning that incorporates human perceptual structure during the initialization phase rather than as a downstream fine-tuning step. By integrating human-derived triplet embeddings from the NIGHTS dataset to initialize a CLIP vision encoder, followed by self-supervised learning on YFCC15M, our approach demonstrates significant zero-shot performance improvements, without any task-specific fine-tuning, across 29 zero shot classification and 2 retrieval benchmarks. On ImageNet-1K, zero-shot gains emerge after approximately 15 epochs of pretraining. Benefits are observed across datasets of various scales, with improvements manifesting at different stages of the pretraining process depending on dataset characteristics. Our approach consistently enhances zero-shot top-1 accuracy, top-5 accuracy, and retrieval recall (e.g., R@1, R@5) across these diverse evaluation tasks, without requiring any adaptation to target domains. These findings challenge the conventional wisdom of using human-perceptual data primarily for fine-tuning and demonstrate that embedding human perceptual structure during early representation learning yields more capable and vision-language aligned systems that generalize immediately to unseen tasks. Our work shows that "beginning with you", starting with human perception, provides a stronger foundation for general-purpose vision-language intelligence.
- Open Access
The brain dynamics of congenitally blind people seeing faces and letters via sound
Summary: Sensory substitution devices (SSDs) convert images to sounds to equip blind individuals with nominally visual functions, like face or letter sensitivity. Prior studies showed that image-to-sound SSDs engage cortices ordinarily specialised for visual functions. However, the brain dynamics of SSD-supported perception remains unknown. Either visual cortices are the first locus of discrimination of SSD percepts, or their activation is a byproduct of perceptual processes unfurling elsewhere, such as in auditory cortices. Resolving this uncertainty is critical for understanding the contribution of visual cortices to mechanisms subserving SSD-induced perceptions in the blind. Using electrical neuroimaging of EEG data from congenitally blind adults, we show for the first time that visual cortices are the earliest site of face and letter sensitivity when conveyed via SSDs, though with distinct temporal dynamics and spatial localizations. In the case of faces versus scrambled faces, differences first manifested at 460ms post-stimulus onset as topographic EEG modulations that were in turn localized to a network of right-hemisphere regions, including the fusiform face area (FFA) as well as lateral occipital cortices. In the case of discerning vertically versus horizontally oriented letter shapes, differences first manifested at 370ms post-stimulus onset as topographic EEG modulations that were localized to a network of left-hemisphere regions, including the occipital pole and the occipito-temporal junction extending into the angular gyrus. Notably, early-latency auditory evoked potentials did not differ based on visual properties of the soundscapes. These collective data support the proposition that responses to soundscapes are routed to brain circuits ordinarily specialized for processing visual information and that these circuits are the loci of the initial discriminant processes. By providing the temporal dynamics of SSD perception, our findings provide unique evidence for the theory that cortices are characterised by task-contingent functional organisation. Highlights: Sensory substitution devices (SSDs) provide the blind access to visual information Brain dynamics in visual cortices to SSD-conveyed object images are unknown Early EEG sensitivity for faces and letters was localized to the visual cortex The blind visual cortex takes part in early SSD processing for visual categories eTOC blurb: We characterise for the first time the brain dynamics of the blind using soundscapes for face and letter discrimination. Faces recruited the FFA at 460ms post-stimulus onset, whereas letters recruited the occipital pole, occipito-temporal junction, and angular gyrus at 370ms post-stimulus onset. Our findings of the temporal dynamics of SSD perception uniquely support the theory that cortices are characterised by task-selective sensory-independent functional organisation.
Open AccessThe brain dynamics of congenitally blind people seeing faces and letters via sound
Summary: Sensory substitution devices (SSDs) convert images to sounds to equip blind individuals with nominally visual functions, like face or letter sensitivity. Prior studies showed that image-to-sound SSDs engage cortices ordinarily specialised for visual functions. However, the brain dynamics of SSD-supported perception remains unknown. Either visual cortices are the first locus of discrimination of SSD percepts, or their activation is a byproduct of perceptual processes unfurling elsewhere, such as in auditory cortices. Resolving this uncertainty is critical for understanding the contribution of visual cortices to mechanisms subserving SSD-induced perceptions in the blind. Using electrical neuroimaging of EEG data from congenitally blind adults, we show for the first time that visual cortices are the earliest site of face and letter sensitivity when conveyed via SSDs, though with distinct temporal dynamics and spatial localizations. In the case of faces versus scrambled faces, differences first manifested at 460ms post-stimulus onset as topographic EEG modulations that were in turn localized to a network of right-hemisphere regions, including the fusiform face area (FFA) as well as lateral occipital cortices. In the case of discerning vertically versus horizontally oriented letter shapes, differences first manifested at 370ms post-stimulus onset as topographic EEG modulations that were localized to a network of left-hemisphere regions, including the occipital pole and the occipito-temporal junction extending into the angular gyrus. Notably, early-latency auditory evoked potentials did not differ based on visual properties of the soundscapes. These collective data support the proposition that responses to soundscapes are routed to brain circuits ordinarily specialized for processing visual information and that these circuits are the loci of the initial discriminant processes. By providing the temporal dynamics of SSD perception, our findings provide unique evidence for the theory that cortices are characterised by task-contingent functional organisation. Highlights: Sensory substitution devices (SSDs) provide the blind access to visual information Brain dynamics in visual cortices to SSD-conveyed object images are unknown Early EEG sensitivity for faces and letters was localized to the visual cortex The blind visual cortex takes part in early SSD processing for visual categories eTOC blurb: We characterise for the first time the brain dynamics of the blind using soundscapes for face and letter discrimination. Faces recruited the FFA at 460ms post-stimulus onset, whereas letters recruited the occipital pole, occipito-temporal junction, and angular gyrus at 370ms post-stimulus onset. Our findings of the temporal dynamics of SSD perception uniquely support the theory that cortices are characterised by task-selective sensory-independent functional organisation.
- Open Access
- 19 November 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 43 (6) , 1-10
Identifying Behavioral Correlates to Visual Discomfort
Outside of self-report surveys, there are no proven, reliable methods to quantify visual discomfort or visually induced motion sickness symptoms when using head-mounted displays. While valuable tools, self-report surveys suffer from potential biases and low sensitivity due to variability in how respondents may assess and report their experience. Consequently, extreme visual-vestibular conflicts are generally used to induce discomfort symptoms large enough to measure reliably with surveys (e.g., stationary participants riding virtual roller coasters). An emerging area of research is the prediction of discomfort survey results from physiological and behavioral markers. However, the signals derived from experimental paradigms that are explicitly designed to be uncomfortable may not generalize to more naturalistic experiences where comfort is prioritized. In this work we introduce a custom VR headset designed to introduce significant near-eye optical distortion (i.e., pupil swim) to induce visual discomfort during more typical VR experiences. We evaluate visual comfort in our headset while users play the popular VR title Job Simulator and show that eye-tracked dynamic distortion correction improves visual comfort in a multi-session, within-subjects user study. We additionally use representational similarity analysis to highlight changes in head and gaze behavior that are potentially more sensitive to visual discomfort than surveys.
- 19 November 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 43 (6) , 1-10
Open AccessIdentifying Behavioral Correlates to Visual Discomfort
Outside of self-report surveys, there are no proven, reliable methods to quantify visual discomfort or visually induced motion sickness symptoms when using head-mounted displays. While valuable tools, self-report surveys suffer from potential biases and low sensitivity due to variability in how respondents may assess and report their experience. Consequently, extreme visual-vestibular conflicts are generally used to induce discomfort symptoms large enough to measure reliably with surveys (e.g., stationary participants riding virtual roller coasters). An emerging area of research is the prediction of discomfort survey results from physiological and behavioral markers. However, the signals derived from experimental paradigms that are explicitly designed to be uncomfortable may not generalize to more naturalistic experiences where comfort is prioritized. In this work we introduce a custom VR headset designed to introduce significant near-eye optical distortion (i.e., pupil swim) to induce visual discomfort during more typical VR experiences. We evaluate visual comfort in our headset while users play the popular VR title Job Simulator and show that eye-tracked dynamic distortion correction improves visual comfort in a multi-session, within-subjects user study. We additionally use representational similarity analysis to highlight changes in head and gaze behavior that are potentially more sensitive to visual discomfort than surveys.
- Open Access
- 30 April 2024
- Published by Frontiers Media SA in Frontiers in Neuroscience
- Vol. 18, 1396101
Assessing translational applicability of perineuronal net dysfunction in Alzheimer’s disease across species
In the context of aging and age-associated neurodegenerative disorders, the brain’s extracellular matrix (ECM) serves as a critical regulator for neuronal health and cognitive function. Within the extracellular space, proteoglycans and their glycosaminoglycan attachments play essential roles in forming, stabilizing, and protecting neural circuits throughout neurodevelopment and adulthood. Recent studies in rodents reveal that chondroitin sulfate-glycosaminoglycan (CS-GAG) containing perineuronal nets (PNNs) exhibit both structural and compositional differences throughout the brain. While animal studies are illuminating, additional research is required to translate these interregional PNN/CS-GAG variations to human brain tissue. In this perspective article, we first investigate the translational potential for interregional CS-GAG variances across species as novel targets for region-specific therapeutic development. We specifically focus on the observation that alterations in brain PNN-associated CS-GAGs have been linked with the progression of Alzheimer’s disease (AD) neuropathology in humans, but these changes have not been fully recapitulated in rodent models of this disease. A second highlight of this perspective article investigates whether AD-associated shifts in CS-GAGs in humans may be dependent on region-specific baseline differences in CS-GAG sulfation patterning. The current findings begin to disentangle the intricate relationships between the interregional differences in brain PNN/CS-GAG matrices across species, while emphasizing the need to better understand the close relationship between dementia and changes in brain CS-GAG sulfation patterns in patients with AD and related dementias.
- 30 April 2024
- Published by Frontiers Media SA in Frontiers in Neuroscience
- Vol. 18, 1396101
Open AccessAssessing translational applicability of perineuronal net dysfunction in Alzheimer’s disease across species
In the context of aging and age-associated neurodegenerative disorders, the brain’s extracellular matrix (ECM) serves as a critical regulator for neuronal health and cognitive function. Within the extracellular space, proteoglycans and their glycosaminoglycan attachments play essential roles in forming, stabilizing, and protecting neural circuits throughout neurodevelopment and adulthood. Recent studies in rodents reveal that chondroitin sulfate-glycosaminoglycan (CS-GAG) containing perineuronal nets (PNNs) exhibit both structural and compositional differences throughout the brain. While animal studies are illuminating, additional research is required to translate these interregional PNN/CS-GAG variations to human brain tissue. In this perspective article, we first investigate the translational potential for interregional CS-GAG variances across species as novel targets for region-specific therapeutic development. We specifically focus on the observation that alterations in brain PNN-associated CS-GAGs have been linked with the progression of Alzheimer’s disease (AD) neuropathology in humans, but these changes have not been fully recapitulated in rodent models of this disease. A second highlight of this perspective article investigates whether AD-associated shifts in CS-GAGs in humans may be dependent on region-specific baseline differences in CS-GAG sulfation patterning. The current findings begin to disentangle the intricate relationships between the interregional differences in brain PNN/CS-GAG matrices across species, while emphasizing the need to better understand the close relationship between dementia and changes in brain CS-GAG sulfation patterns in patients with AD and related dementias.
- Open Access
AI Literature Review Suite
The process of conducting literature reviews is often time-consuming and labor-intensive. To streamline this process, I present an AI Literature Review Suite that integrates several functionalities to provide a comprehensive literature review. This tool leverages the power of open access science, large language models (LLMs) and natural language processing to enable the searching, downloading, and organizing of PDF files, as well as extracting content from articles. Semantic search queries are used for data retrieval, while text embeddings and summarization using LLMs present succinct literature reviews. Interaction with PDFs is enhanced through a user-friendly graphical user interface (GUI). The suite also features integrated programs for bibliographic organization, interaction and query, and literature review summaries. This tool presents a robust solution to automate and optimize the process of literature review in academic and industrial research.
Open AccessAI Literature Review Suite
The process of conducting literature reviews is often time-consuming and labor-intensive. To streamline this process, I present an AI Literature Review Suite that integrates several functionalities to provide a comprehensive literature review. This tool leverages the power of open access science, large language models (LLMs) and natural language processing to enable the searching, downloading, and organizing of PDF files, as well as extracting content from articles. Semantic search queries are used for data retrieval, while text embeddings and summarization using LLMs present succinct literature reviews. Interaction with PDFs is enhanced through a user-friendly graphical user interface (GUI). The suite also features integrated programs for bibliographic organization, interaction and query, and literature review summaries. This tool presents a robust solution to automate and optimize the process of literature review in academic and industrial research.
- Open Access
Near-field potentials index local neural computations more accurately than population spiking
Local field potentials (LFP) are low-frequency extracellular voltage fluctuations thought to primarily arise from synaptic activity. However, unlike highly localized neuronal spiking, LFP is spatially less specific. LFP measured at one location is not entirely generated there due to far-field contributions that are passively conducted across volumes of neural tissue. We sought to quantify how much information within the locally generated, near-field low-frequency activity (nfLFP) is masked by volume-conducted far-field signals. To do so, we measured laminar neural activity in primary visual cortex (V1) of monkeys viewing sequences of multifeatured stimuli. We compared information content of regular LFP and nfLFP that was mathematically stripped of volume-conducted far-field contributions. Information content was estimated by decoding stimulus properties from neural responses via spatiotemporal multivariate pattern analysis. Volume-conducted information differed from locally generated information in two important ways: (1) for stimulus features relevant to V1 processing (orientation and eye-of-origin), nfLFP contained more information. (2) in contrast, the volume-conducted signal was more informative regarding temporal context (relative stimulus position in a sequence), a signal likely to be coming from elsewhere. Moreover, LFP and nfLFP differed both spectrally as well as spatially, urging caution regarding the interpretations of individual frequency bands and/or laminar patterns of LFP. Most importantly, we found that population spiking of local neurons was less informative than either the LFP or nfLFP, with nfLFP containing most of the relevant information regarding local stimulus processing. These findings suggest that the optimal way to read out local computational processing from neural activity is to decode the local contributions to LFP, with significant information loss hampering both regular LFP and local spiking. Author’s Contributions Conceptualization, D.A.T., J.A.W, and A.M.; Data Collection, J.A.W., M.A.C., K.D.; Formal Analysis, D.A.T. and J.A.W.; Data Visualization, D.A.T. and J.A.W.; Original Draft, D.A.T., J.A.W., and A.M.; Revisions and Final Draft, D.A.T., J.A.W., M.A.C., K.D., M.T.W., A.M.B., and A.M. Competing Interests The authors declare no conflicts of interest.
Open AccessNear-field potentials index local neural computations more accurately than population spiking
Local field potentials (LFP) are low-frequency extracellular voltage fluctuations thought to primarily arise from synaptic activity. However, unlike highly localized neuronal spiking, LFP is spatially less specific. LFP measured at one location is not entirely generated there due to far-field contributions that are passively conducted across volumes of neural tissue. We sought to quantify how much information within the locally generated, near-field low-frequency activity (nfLFP) is masked by volume-conducted far-field signals. To do so, we measured laminar neural activity in primary visual cortex (V1) of monkeys viewing sequences of multifeatured stimuli. We compared information content of regular LFP and nfLFP that was mathematically stripped of volume-conducted far-field contributions. Information content was estimated by decoding stimulus properties from neural responses via spatiotemporal multivariate pattern analysis. Volume-conducted information differed from locally generated information in two important ways: (1) for stimulus features relevant to V1 processing (orientation and eye-of-origin), nfLFP contained more information. (2) in contrast, the volume-conducted signal was more informative regarding temporal context (relative stimulus position in a sequence), a signal likely to be coming from elsewhere. Moreover, LFP and nfLFP differed both spectrally as well as spatially, urging caution regarding the interpretations of individual frequency bands and/or laminar patterns of LFP. Most importantly, we found that population spiking of local neurons was less informative than either the LFP or nfLFP, with nfLFP containing most of the relevant information regarding local stimulus processing. These findings suggest that the optimal way to read out local computational processing from neural activity is to decode the local contributions to LFP, with significant information loss hampering both regular LFP and local spiking. Author’s Contributions Conceptualization, D.A.T., J.A.W, and A.M.; Data Collection, J.A.W., M.A.C., K.D.; Formal Analysis, D.A.T. and J.A.W.; Data Visualization, D.A.T. and J.A.W.; Original Draft, D.A.T., J.A.W., and A.M.; Revisions and Final Draft, D.A.T., J.A.W., M.A.C., K.D., M.T.W., A.M.B., and A.M. Competing Interests The authors declare no conflicts of interest.
- Open Access
Getting the gist faster: Blurry images enhance the early temporal similarity between neural signals and convolutional neural networks
Humans are able to recognize objects under a variety of noisy conditions, so models of the human visual system must account for how this feat is accomplished. In this study, we investigated how image perturbations, specifically reducing images to their low spatial frequency (LSF) components, affected correspondence between convolutional neural networks (CNNs) and brain signals recorded using magnetoencephalography (MEG). Using the high temporal resolution of MEG, we found that CNN-Brain correspondence for deeper and more complex layers across CNN architectures emerged earlier for LSF images than for their unfiltered broadband counterparts. The early emergence of LSF components is consistent with the coarse-to-fine theoretical framework for visual image processing, but surprisingly shows that LSF signals from images are more prominent when high spatial frequencies are removed. In addition, we decomposed MEG signals into oscillatory components and found correspondence varied based on frequency bands, painting a full picture of how CNN-Brain correspondence varies with time, frequency, and MEG sensor locations. Finally, we varied image properties of CNN training sets, and found marked changes in CNN processing dynamics and correspondence to brain activity. In sum, we show that image perturbations affect CNN-Brain correspondence in unexpected ways, as well as provide a rich methodological framework for assessing CNN-Brain correspondence across space, time, and frequency.
Open AccessGetting the gist faster: Blurry images enhance the early temporal similarity between neural signals and convolutional neural networks
Humans are able to recognize objects under a variety of noisy conditions, so models of the human visual system must account for how this feat is accomplished. In this study, we investigated how image perturbations, specifically reducing images to their low spatial frequency (LSF) components, affected correspondence between convolutional neural networks (CNNs) and brain signals recorded using magnetoencephalography (MEG). Using the high temporal resolution of MEG, we found that CNN-Brain correspondence for deeper and more complex layers across CNN architectures emerged earlier for LSF images than for their unfiltered broadband counterparts. The early emergence of LSF components is consistent with the coarse-to-fine theoretical framework for visual image processing, but surprisingly shows that LSF signals from images are more prominent when high spatial frequencies are removed. In addition, we decomposed MEG signals into oscillatory components and found correspondence varied based on frequency bands, painting a full picture of how CNN-Brain correspondence varies with time, frequency, and MEG sensor locations. Finally, we varied image properties of CNN training sets, and found marked changes in CNN processing dynamics and correspondence to brain activity. In sum, we show that image perturbations affect CNN-Brain correspondence in unexpected ways, as well as provide a rich methodological framework for assessing CNN-Brain correspondence across space, time, and frequency.