Ruey-Song Huang

Functional magnetic resonance imaging (fMRI) has been widely used to explore the neural mechanisms underlying speech processing. However, the intertwining of perception and production that exists in real-world scenarios remains underexplored due to challenges such as gradient noise and head motion artifacts from speaking. Previous research has often employed sparse-sampling designs, pausing image acquisition intermittently to present auditory stimuli or record overt speech. While this approach mitigates some challenges, it cannot capture continuous brain activity during speech processing and does not separate the mixed hemodynamic responses to external and self-generated speech occurring in succession. We overcame these limitations and continuously scanned 31 participants as they listened to and recited English sentences. Through independent component analysis (ICA), we decomposed each functional scan into spatially independent components (ICs), identifying task-related ICs with activations predominantly located in the superior temporal cortex, inferior frontal gyrus, and orofacial sensorimotor cortex. These ICs demonstrated time-resolved hemodynamic responses corresponding to distinct stages of speech perception, planning, and production. A linear subtraction between the IC time courses from the listening-reciting (perception-to-production) and listening (perception-only) tasks further revealed a secondary hemodynamic response to self-generated speech in the superior temporal cortex. Furthermore, we established precise temporal relationships between overt speech output and the peak, rise, and fall of hemodynamic responses for each independent component. Together, we present a methodological framework that can inform future fMRI studies on naturalistic tasks involving the perception of external auditory stimuli and monitoring of self-generated sounds.

Modern machine learning models leveraging multi-omics data face significant privacy challenges due to the sensitive nature of patient information. Communication overhead and missing features in each institution can lead to a substantial decline in federated learning performance. In response to these concerns, we propose VFLING-Federated Learning for Multi-Omics Data Integration with Graphs-a secure one-shot communication federated learning framework. We note that medical data reflects disease characteristics from different omics, while the relationships between samples exhibit relative stability across these omics. To minimize data transmission while maximizing the utilization of each participant's feature information we develop a strategy that transmits not only the local features but also the relationships or topology in one-shot communication. By fusing the omics based on the locally learned graph structure instead of features, VFLING enables improved performance even when some features are missing from individual parties. Extensive experiments demonstrate that VFLING outperforms existing frameworks, paving the way for applications in the medical field. Local features and graph topology are shared to the trainable server in a single communication, enhancing model accuracy through integrated data. This approach improves robustness despite incomplete information. Local Parties and Server Integration: Local parties learn and transmit both local features and graph topology to the server in a single communication, maximizing effective information transfer. The trainable server then integrates data across parties using graph relationships, enhancing model robustness and accuracy despite incomplete feature sets.

Functional magnetic resonance imaging (fMRI) has been widely utilized to explore the neural mechanisms underlying speech processing. However, the intertwining of perception and production that exists in real-world scenarios remains underexplored due to challenges such as gradient noise and head motion artifacts from speaking. Previous research has often employed sparse-sampling designs, pausing image acquisition intermittently to present auditory stimuli or record overt speech. While this approach mitigates some challenges, it cannot capture continuous brain activity during speech processing and does not separate the mixed hemodynamic responses to external and self-generated speech occurring in succession. We overcame these limitations and continuously scanned thirty-one participants as they listened to and recited English sentences. Through independent component analysis (ICA), we decomposed each functional scan into spatially independent components (ICs), identifying task-related ICs in the superior temporal cortex, inferior frontal gyrus, and orofacial sensorimotor cortex. These ICs demonstrated time-resolved hemodynamic responses corresponding to distinct stages of speech perception, planning, and production. A linear subtraction between the IC time courses from the listening-reciting (perception-to-production) and listening (perception-only) tasks further revealed a secondary hemodynamic response to self-generated speech in the superior temporal cortex. Furthermore, we established precise temporal relationships between overt speech output and the peak, rise, and fall of hemodynamic responses for each independent component. Together, we present a methodological framework that can inform future fMRI studies on naturalistic tasks involving the perception of external auditory stimuli and monitoring of self-generated sounds.

A central question in bilingualism research is whether challenges in a second language (L2) reflect a fundamentally different processing route from the first language (L1), or slower processing at particular stages. Although functional magnetic resonance imaging (fMRI) revealed substantial cortical overlap between L1 and L2, different processing mechanisms may interleave within the overlap. Conventional fMRI methods lack the combined spatial and temporal resolution needed to determine precisely where and when L1 and L2 dissociations occur. Here, we used rapid phase-encoded fMRI to track L1 and L2 processing within overlapping cortical regions. Thirty-one Chinese-English bilinguals completed sentence-level reading, listening, reading-aloud, and shadowing tasks in both Chinese (L1) and English (L2). By analyzing hemodynamic traveling waves along sample paths through the visual, auditory, and motor systems, we identified temporal dissociations between L1 and L2 processing. Compared with L1, L2 exhibited significant and consistent delays along dorsal and ventral visual streams during reading and reading aloud. Along auditory streams, L2 delays differed between externally presented speech (listening) and self-generated speech (reading aloud). In the frontal cortex, reading aloud and shadowing exhibited similar progression patterns, but with different temporal offsets between L1 and L2 in the dorsal and ventral motor streams. These findings reveal task- and modality-dependent differences between L1 and L2 processing, and pinpoint the exact locations for delayed L2 processing across multimodal streams. This study suggests that L2 disadvantage is often a matter of timing and efficiency within largely shared brain networks, rather than due to the use of different brain regions. Significant Statement: Rapid phase-encoded fMRI introduces a temporal dimension for comparing L1 and L2 processing, tracking the precise timing of neural information flow across the brain. Moving beyond functional localization, we directly visualize and quantify the propagation of hemodynamic traveling waves along selected paths in visual, auditory, and motor systems. Our ability to characterize temporal delays along specific functional streams represents an advancement in understanding dynamic cognitive processes beyond language. The findings can influence theories of language acquisition and cognitive processing in bilingual individuals. The methods to pinpoint locations where delays occur can be applied to other areas of cognitive neuroscience, allowing the exploration of complex neural interaction in real time across sensory, motor, and cognitive domains.

Language and music, two core forms of human communication, both rely on rhythmic control. To date, the neural ensembles supporting their real-time production are not fully characterized. Using rapid phase-encoded fMRI, we mapped the spatiotemporal brain dynamics of overt language and music production under varying rhythm control demands. We observed overlapping hemodynamic traveling waves across domains, with auditory-motor regions activated earlier, followed by visual, posterior parietal, and Sylvian parietal-temporal (Spt) regions, supporting sensorimotor transformation and online monitoring of production. Increasing rhythmic complexity elicited slower and stronger activations in both domains: compared with baseline, regular rhythm control elicited delayed and amplified responses, while irregular rhythm control produced the slowest and strongest responses. During music production, rhythm-related activity expanded into frontal and temporal language regions, suggesting that higher rhythmic demands engage language resources. These findings reveal dynamic neural ensembles that can flexibly reconfigure shared resources to support temporal precision in sequence production.

This study investigates the intricate interplay between the task-positive network and the default mode network (DMN) during transitions between overt language tasks and brief resting periods. While previous research suggests that these networks are not invariably anticorrelated, the precise timing of transitions has remained elusive. We employed rapid phase-encoded fMRI to decode brain dynamics with ultimate precision, capturing these transitions in real time. By utilizing phasor diagrams to represent the oscillatory activities, we examined the amplitudes and phases of hemodynamic fluctuations within the language network and DMN. Our findings align with existing empirical and theoretical perspectives on DMN functions and cognitive task performance, affirming the validity of our approach. We identified heterogeneous micro resting states interwoven with periods of overt speech production. Notably, various core regions of the DMN exhibited task-dependent amplitude and phase modulations, with activation strength and delay rising in line with increasing task complexity, ranging from comprehension to immediate and delayed speech production. This study sheds light on the dynamic engagement of the DMN during overt speech production, providing precise timing data of transitions between the DMN and language network. It demonstrates that rapid phase-encoded fMRI and phasor diagrams are powerful tools for measuring the switching between active tasks and micro resting states with subsecond accuracy, while also elucidating task load-dependent changes in the DMN. By accurately measuring the timing of these transitions, we gain insights into cognitive flexibility, attention, and the efficiency of information processing.

Background and aims: Clinical diagnosis of gambling disorder (GD) remains challenging due to the heterogeneity in symptoms and a lack of consistency in the proposed neural mechanisms. Effective classification of GD may depend on neural representations of either risky decision-making or reward processing. Methods: To address these challenges, we recruited more than 100 individuals with GD and matched healthy controls, utilizing event-related fMRI during a novel risky decision-making task to elicit neural representations of risky decision-making and reward processing. Results: During the decision phase, there was no significant difference observed between the two groups even when a very liberal threshold was used. During reward processing, the GD group exhibited significantly increased activation in the right inferior frontal gyrus, right anterior insula, and bilateral posterior cingulate cortex in the risky reward condition compared with the healthy controls. A notable neural activation characteristic was the distinct response between risk-win and risk-loss conditions in reward processing, particularly in the right inferior frontal gyrus in the GD group. The classification for GD using the neural representation of reward yielded an area under the curve of 0.75 (±0.11 SD). Discussion and conclusion: These findings integrate biological and behavioral perspectives to provide new insights into the reward processes underlying GD. These findings highlight specific neural representations associated with GD and suggest potential biomarkers for diagnostic evaluation in GD.

What takes precedence in our brain: music or language? Here we used rapid phase-encoded fMRI to explore spatiotemporal brain dynamics during naturalistic music and language tasks, seeking to track down the entire perception-to-production process and compare the speed of information transfer between the two domains. We found largely shared streams of traveling waves bilaterally, along which musical codes were transported faster than language codes for identical visual input.

This study explored the relationship between multifaceted multilingualism and cognitive shifting through a task-switching paradigm using fMRI. Multilingualism was modeled from both convergent (i.e., integrated multilingual index) and divergent (i.e., L2 proficiency, interpreting training, language entropy) perspectives. Participants identified letters or numbers based on task cues, with Repeat trials maintaining the same task and Switch trials requiring a different task. Switch cost (Switch–Repeat) was used to reflect shifting demands. Better task-switching performance was associated with a higher integrated multilingual index and interpreting training. Neuroimaging indicated that multilinguals predominantly engaged left-hemisphere regions for switching, with extensive multilingual experience requiring fewer neural resources for switch cost (i.e., more efficient processing for cognitive control). During task switching, brain connectivity was regulated locally by L2 proficiency, and globally by interpreting training. These findings underscore the importance of considering multifaceted multilingual experience to understand its impact on cognitive function and brain activity.

Natural language processing unfolds information overtime as spatially separated, multimodal, and interconnected neural processes. Existing noninvasive subtraction‐based neuroimaging techniques cannot simultaneously achieve the spatial and temporal resolutions required to visualize ongoing information flows across the whole brain. Here we have developed rapid phase‐encoded designs to fully exploit the temporal information latent in functional magnetic resonance imaging data, as well as overcoming scanner noise and head‐motion challenges during overt language tasks. We captured real‐time information flows as coherent hemodynamic waves traveling over the cortical surface during listening, reading aloud, reciting, and oral cross‐language interpreting tasks. We were able to observe the timing, location, direction, and surge of traveling waves in all language tasks, which were visualized as “brainstorms” on brain “weather” maps. The paths of hemodynamic traveling waves provide direct evidence for dual‐stream models of the visual and auditory systems as well as logistics models for crossmodal and cross‐language processing. Specifically, we have tracked down the step‐by‐step processing of written or spoken sentences first being received and processed by the visual or auditory streams, carried across language and domain‐general cognitive regions, and finally delivered as overt speeches monitored through the auditory cortex, which gives a complete picture of information flows across the brain during natural language functioning.Practitioner Points: Phase‐encoded fMRI enables simultaneous imaging of high spatial and temporal resolution, capturing continuous spatiotemporal dynamics of the entire brain during real‐time overt natural language tasks. Spatiotemporal traveling wave patterns provide direct evidence for constructing comprehensive and explicit models of human information processing. This study unlocks the potential of applying rapid phase‐encoded fMRI to indirectly track the underlying neural information flows of sequential sensory, motor, and high‐order cognitive processes.