Multisensory naturalistic decoding with high-density diffuse optical tomography
Authors:
- Kalyan TripathyKTKalyan Tripathy
- Zachary E. MarkowZMZachary E. Markow
- Morgan FogartyMFMorgan Fogarty
- Mariel L. SchroederMSMariel L. Schroeder
- Alexa M. SvobodaASAlexa M. Svoboda
- Adam T. EggebrechtAEAdam T. Eggebrecht
- Bradley L. SchlaggarBSBradley L. Schlaggar
- Jason W. TrobaughJTJason W. Trobaugh
- Joseph P. CulverJCJoseph P. Culver
Open Access
- 1 January 2025
- journal article
- Published by SPIE-Intl Soc Optical Eng in Neurophotonics
- Vol. 12 (01) , 015002-16
Abstract
Decoding of naturalistic information from functional brain imaging data acquired in natural settings has neuroscientific importance for understanding cortical function1 – 3 and clinical implications for patients with neurological conditions affecting speech and movement—from cerebral palsy to strokes and neurodegenerative disorders.4,5 Studies using recording methods such as functional magnetic resonance imaging (fMRI) and electrocorticography (ECoG) have accurately reconstructed detailed visual and auditory information from cortical brain activity.3,4,6 For instance, visual cortex activity sampled with fMRI has been used to reconstruct images and movies, providing insight into how these types of information are encoded in the brain and can potentially be extracted to recreate memories, dreams, and real or imagined scenes.3,7 – 10 Furthermore, fMRI-recorded responses to narrative stories have been used to map semantic categories across the cortex and accurately reconstruct semantic representations of continuous language.6,11 Meanwhile, intelligible speech has been synthesized from ECoG data,4,12 and accurate and efficient writing has been achieved by decoding intracortical microelectrode recordings,13 with promise for patients who are otherwise unable to communicate due to severe speech and motor disorders. However, the widespread application of these findings is limited by the recording methods used. The logistics of MRI scanners are not conducive to purposes such as daily communication, whereas ECoG and intracortical microelectrodes require invasive neurosurgery. To address these limitations, decoding has been investigated using non-invasive and portable technologies such as electroencephalography14 – 16 and functional near-infrared spectroscopy (fNIRS)17 – 23 with promising results. However, the traditionally low spatial resolutions of these modalities present a challenge for the nuanced classification of complex naturalistic information. For instance, traditional fNIRS has enabled fascinating decoding research in infants, clinical populations, and brain–computer interfaces but has generally been limited by image quality to coarse classifications between two and six targets.14 – 16,19,24,25 Applications of naturalistic decoding could benefit greatly from high-resolution brain imaging approaches that can be employed in natural settings.Keywords
This publication has 52 references indexed in Scilit:
- Modeling the hemodynamic response function in fMRI: Efficiency, bias and mis-modelingNeuroImage, 2009
- "Who" Is Saying "What"? Brain-Based Decoding of Human Voice and SpeechScience, 2008
- Mechanisms of Face PerceptionAnnual Review of Neuroscience, 2008
- Identifying natural images from human brain activityNature, 2008
- Retinotopic mapping of adult human visual cortex with high-density diffuse optical tomographyProceedings of the National Academy of Sciences, 2007
- Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain–computer interfaceNeuroImage, 2007
- Diffuse Optical Tomography of Cerebral Blood Flow, Oxygenation, and Metabolism in Rat during Focal IschemiaJournal of Cerebral Blood Flow & Metabolism, 2003
- Neurophysiological investigation of the basis of the fMRI signalNature, 2001
- A finite element approach for modeling photon transport in tissueMedical Physics, 1993
- Estimation of optical pathlength through tissue from direct time of flight measurementPhysics in Medicine & Biology, 1988