Navigating Brain Language Representations: A Comparative Analysis of Neural Language Models and Psychologically Plausible Models
Preprint
Abstract
Neural language models, particularly large-scale ones, have been consistently proven to be most effective in predicting brain neural activity across a range of studies. However, previous research overlooked the comparison of these models with psychologically plausible ones. Moreover, evaluations were reliant on limited, single-modality, and English cognitive datasets. To address these questions, we conducted an analysis comparing encoding performance of various neural language models and psychologically plausible models. Our study utilized extensive multi-modal cognitive datasets, examining bilingual word and discourse levels. Surprisingly, our findings revealed that psychologically plausible models outperformed neural language models across diverse contexts, encompassing different modalities such as fMRI and eye-tracking, and spanning languages from English to Chinese. Among psychologically plausible models, the one incorporating embodied information emerged as particularly exceptional. This model demonstrated superior performance at both word and discourse levels, exhibiting robust prediction of brain activation across numerous regions in both English and Chinese.Keywords
All Related Versions
This publication has 27 references indexed in Scilit:
- MulCogBench: a multi-modal cognitive benchmark dataset for evaluating Chinese and English computational language modelsLanguage Resources and Evaluation, 2025
- Abstract representations of events arise from mental errors in learning and memoryNature Communications, 2020
- Connecting concepts in the brain by mapping cortical representations of semantic relationsNature Communications, 2020
- Toward a universal decoder of linguistic meaning from brain activationNature Communications, 2018
- Natural speech reveals the semantic maps that tile human cerebral cortexNature, 2016
- Neural representations of events arise from temporal community structureNature Neuroscience, 2013
- Modality-Constrained Statistical Learning of Tactile, Visual, and Auditory Sequences.Journal of Experimental Psychology: Learning, Memory, and Cognition, 2005
- Lexical Access in Chinese Speech Comprehension and ProductionBrain and Language, 1999
- The representation of text-based and situation-based information in discourse comprehensionJournal of Memory and Language, 1990
- Time-locked multiregional retroactivation: A systems-level proposal for the neural substrates of recall and recognitionCognition, 1989