Anwar O. Nunez-Elizalde
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Anwar O. Nunez-Elizalde has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
- Anuja Negi
Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
2 shared publications - Fatma Deniz
Faculty of Electrical Engineering and Computer Science, Technische Universität Berlin, 10623 Berlin, Germany
2 shared publications - Alan Urban
KU Leuven
1 shared publication - Charu Bai Reddy
UCL Institute of Ophthalmology, University College London, London, UK
1 shared publication Christine Tseng
1 shared publication- Gabriel Montaldo
KU Leuven
1 shared publication
Latest Publications
- Open Access
Optimizing Language Model Embeddings to Voxel Activity Improves Brain Activity Predictions
Recent studies have shown that contextual semantic embeddings from language models can accurately predict human brain activity during language processing. However, most studies use contextual embeddings with the same context length and model layer for all voxels, potentially overlooking meaningful variations across the brain. In this study, we investigate whether optimizing contextual embeddings for individual voxels improves their ability to predict brain activity during reading. We optimize embeddings for each voxel by selecting the best-predicting context length, model layer, or both. We perform this optimization with two different types of stimuli (isolated sentences and narratives), and quantify the performance gains of optimized embeddings over standard fixed embeddings. Our results show that voxel-specific optimization substantially improves the prediction accuracy of contextual semantic embeddings. These findings demonstrate that voxel-specific contextual tuning provides a more accurate and nuanced account of how the contextual semantic information is represented across the cortex.
Open AccessOptimizing Language Model Embeddings to Voxel Activity Improves Brain Activity Predictions
Recent studies have shown that contextual semantic embeddings from language models can accurately predict human brain activity during language processing. However, most studies use contextual embeddings with the same context length and model layer for all voxels, potentially overlooking meaningful variations across the brain. In this study, we investigate whether optimizing contextual embeddings for individual voxels improves their ability to predict brain activity during reading. We optimize embeddings for each voxel by selecting the best-predicting context length, model layer, or both. We perform this optimization with two different types of stimuli (isolated sentences and narratives), and quantify the performance gains of optimized embeddings over standard fixed embeddings. Our results show that voxel-specific optimization substantially improves the prediction accuracy of contextual semantic embeddings. These findings demonstrate that voxel-specific contextual tuning provides a more accurate and nuanced account of how the contextual semantic information is represented across the cortex.
- Open Access
Brain-Informed Fine-Tuning for Improved Multilingual Understanding in Language Models
Recent studies have demonstrated that fine-tuning language models with brain data can improve their semantic understanding, although these findings have so far been limited to English. Interestingly, similar to the shared multilingual embedding space of pretrained multilingual language models, human studies provide strong evidence for a shared semantic system in bilingual individuals. Here, we investigate whether fine-tuning language models with bilingual brain data changes model representations in a way that improves them across multiple languages. To test this, we fine-tune monolingual and multilingual language models using brain activity recorded while bilingual participants read stories in English and Chinese. We then evaluate how well these representations generalize to the bilingual participants’ first language, their second language, and several other languages that the participants are not fluent in. We assess the fine-tuned language models on brain encoding performance and downstream NLP tasks. Our results show that bilingual brain-informed fine-tuned language models outperform their vanilla (pretrained) counterparts in both brain encoding performance and most downstream NLP tasks across multiple languages. These findings suggest that brain-informed fine-tuning improves multilingual understanding in language models, offering a bridge between cognitive neuroscience and NLP research. We make our code publicly available. 2
Open AccessBrain-Informed Fine-Tuning for Improved Multilingual Understanding in Language Models
Recent studies have demonstrated that fine-tuning language models with brain data can improve their semantic understanding, although these findings have so far been limited to English. Interestingly, similar to the shared multilingual embedding space of pretrained multilingual language models, human studies provide strong evidence for a shared semantic system in bilingual individuals. Here, we investigate whether fine-tuning language models with bilingual brain data changes model representations in a way that improves them across multiple languages. To test this, we fine-tune monolingual and multilingual language models using brain activity recorded while bilingual participants read stories in English and Chinese. We then evaluate how well these representations generalize to the bilingual participants’ first language, their second language, and several other languages that the participants are not fluent in. We assess the fine-tuned language models on brain encoding performance and downstream NLP tasks. Our results show that bilingual brain-informed fine-tuned language models outperform their vanilla (pretrained) counterparts in both brain encoding performance and most downstream NLP tasks across multiple languages. These findings suggest that brain-informed fine-tuning improves multilingual understanding in language models, offering a bridge between cognitive neuroscience and NLP research. We make our code publicly available. 2
- Open Access
Feature-space selection with banded ridge regression
Encoding models provide a powerful framework to identify the information represented in brain recordings. In this framework, a stimulus representation is expressed within a feature space and is used in a regularized linear regression to predict brain activity. To account for a potential complementarity of different feature spaces, a joint model is fit on multiple feature spaces simultaneously. To adapt regularization strength to each feature space, ridge regression is extended to banded ridge regression, which optimizes a different regularization hyperparameter per feature space. The present paper proposes a method to decompose over feature spaces the variance explained by a banded ridge regression model. It also describes how banded ridge regression performs a feature-space selection, effectively ignoring non-predictive and redundant feature spaces. This feature-space selection leads to better prediction accuracy and to better interpretability. Banded ridge regression is then mathematically linked to a number of other regression methods with similar feature-space selection mechanisms. Finally, several methods are proposed to address the computational challenge of fitting banded ridge regressions on large numbers of voxels and feature spaces. All implementations are released in an open-source Python package called Himalaya.
Open AccessFeature-space selection with banded ridge regression
Encoding models provide a powerful framework to identify the information represented in brain recordings. In this framework, a stimulus representation is expressed within a feature space and is used in a regularized linear regression to predict brain activity. To account for a potential complementarity of different feature spaces, a joint model is fit on multiple feature spaces simultaneously. To adapt regularization strength to each feature space, ridge regression is extended to banded ridge regression, which optimizes a different regularization hyperparameter per feature space. The present paper proposes a method to decompose over feature spaces the variance explained by a banded ridge regression model. It also describes how banded ridge regression performs a feature-space selection, effectively ignoring non-predictive and redundant feature spaces. This feature-space selection leads to better prediction accuracy and to better interpretability. Banded ridge regression is then mathematically linked to a number of other regression methods with similar feature-space selection mechanisms. Finally, several methods are proposed to address the computational challenge of fitting banded ridge regressions on large numbers of voxels and feature spaces. All implementations are released in an open-source Python package called Himalaya.
- Open Access
Neural correlates of blood flow measured by ultrasound
Summary: Functional ultrasound imaging (fUSI) is a popular method for measuring blood flow and thus infer brain activity, but it relies on the physiology of neurovascular coupling and requires extensive signal processing. To establish to what degree its trial-by-trial signals reflect neural activity, we performed simultaneous fUSI and neural recordings with Neuropixels probes in awake mice. fUSI signals strongly correlated with the slow (<0.3 Hz) fluctuations in local firing rate, and were closely predicted by the smoothed firing rate of local neurons, particularly putative inhibitory neurons. The optimal smoothing filter had width ~3 s, matched the hemodynamic response function of awake mouse, was invariant across mice and stimulus conditions, and similar in cortex and hippocampus. fUSI signals also matched neural firing spatially: firing rates were as highly correlated across hemispheres as fUSI signals. Thus, hemodynamic signals measured by ultrasound bear a simple and accurate relationship to neuronal firing.
Open AccessNeural correlates of blood flow measured by ultrasound
Summary: Functional ultrasound imaging (fUSI) is a popular method for measuring blood flow and thus infer brain activity, but it relies on the physiology of neurovascular coupling and requires extensive signal processing. To establish to what degree its trial-by-trial signals reflect neural activity, we performed simultaneous fUSI and neural recordings with Neuropixels probes in awake mice. fUSI signals strongly correlated with the slow (<0.3 Hz) fluctuations in local firing rate, and were closely predicted by the smoothed firing rate of local neurons, particularly putative inhibitory neurons. The optimal smoothing filter had width ~3 s, matched the hemodynamic response function of awake mouse, was invariant across mice and stimulus conditions, and similar in cortex and hippocampus. fUSI signals also matched neural firing spatially: firing rates were as highly correlated across hemispheres as fUSI signals. Thus, hemodynamic signals measured by ultrasound bear a simple and accurate relationship to neuronal firing.
- Open Access
- 24 August 2018
- Published by The Open Journal in Journal of Open Source Software
- Vol. 3 (28)
cottoncandy: scientific python package for easy cloud storage
Cottoncandy is a python scientific library for storing and accessing numpy array data from cloud-based object stores. cottoncandy obviates the need to write (or read) numpy array data to disk as an intermediate step to storing (or accessing) data in a cloud-based object store (e.g. AWS S3). cottoncandy is a bucket full of syntactic sugar that facilitates the use of cloud storage in typical data science workflows.
- 24 August 2018
- Published by The Open Journal in Journal of Open Source Software
- Vol. 3 (28)
Open Accesscottoncandy: scientific python package for easy cloud storage
Cottoncandy is a python scientific library for storing and accessing numpy array data from cloud-based object stores. cottoncandy obviates the need to write (or read) numpy array data to disk as an intermediate step to storing (or accessing) data in a cloud-based object store (e.g. AWS S3). cottoncandy is a bucket full of syntactic sugar that facilitates the use of cloud storage in typical data science workflows.