Bruno L. Giordano
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Bruno L. Giordano has a total of 144 co-authors. Here are the top co-authors by number of shared publications:
Stephen McAdams
Department of Music Research, Centre for Interdisciplinary Research in Music, Media and Technology, McGill University, Montreal, QC, Canada
19 shared publications- Pascal Belin
Institut de Neurosciences de la Timone, Aix-Marseille-University, Marseille, Provence-Alpes-Côte d'Azur, UMR7289 CNRS, France
10 shared publications - Christoph Kayser
Bielefeld University
6 shared publications - Elia Formisano
Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
6 shared publications - Joachim Gross
Institute of Thermodynamics and Thermal Process Engineering, University of Stuttgart, Pfaffenwaldring 9, 70569 Stuttgart, Germany
6 shared publications - Robin A. A. Ince
Institute of Neuroscience and Psychology, University of Glasgow, Scotland G12 8QB, United Kingdom
5 shared publications
Latest Publications
- Open Access
A behaviourally normed database of 1,377 natural sounds for auditory cognition and neuroscience
Natural-sound research requires stimulus sets that combine acoustic standardization with detailed behavioural characterization. We present 1,377 two-second sounds representing 240 expert-defined source– action classes. We call this database “MaMa Sounds”, as it resulted from the collaborative effort of two academic teams in Maastricht and Marseille. The sounds were manually curated, segmented, sampled at 16 kHz, and labelled with a noun identifying the source and a verb identifying the action. We release deidentified trial-level identification and familiarity data together with multiple per-sound norms (e.g., identification accuracy, confidence and agreement; familiarity), along with overall norms derived with principal component analysis. Noun, verb, and joint noun–verb norms are provided as direct means and medians with the number of contributing observations. This battery preserves process-specific information, while two principal-component scores provide compact overall behavioural-identifiability measures derived from response ease, semantic correspondence, agreement, and familiarity. The repository also contains deterministic response-cleaning code, participant and reference Word2Vec representations, and code reproducing the public sound-level tables. The resource supports stimulus selection, matching, and continuous modelling in auditory cognition and neuroscience.
Open AccessA behaviourally normed database of 1,377 natural sounds for auditory cognition and neuroscience
Natural-sound research requires stimulus sets that combine acoustic standardization with detailed behavioural characterization. We present 1,377 two-second sounds representing 240 expert-defined source– action classes. We call this database “MaMa Sounds”, as it resulted from the collaborative effort of two academic teams in Maastricht and Marseille. The sounds were manually curated, segmented, sampled at 16 kHz, and labelled with a noun identifying the source and a verb identifying the action. We release deidentified trial-level identification and familiarity data together with multiple per-sound norms (e.g., identification accuracy, confidence and agreement; familiarity), along with overall norms derived with principal component analysis. Noun, verb, and joint noun–verb norms are provided as direct means and medians with the number of contributing observations. This battery preserves process-specific information, while two principal-component scores provide compact overall behavioural-identifiability measures derived from response ease, semantic correspondence, agreement, and familiarity. The repository also contains deterministic response-cleaning code, participant and reference Word2Vec representations, and code reproducing the public sound-level tables. The resource supports stimulus selection, matching, and continuous modelling in auditory cognition and neuroscience.
- Open Access
Multiscale Temporal Processing Supports Sound Recognition under Causal Constraints
The auditory system operates under a fundamental computational constraint: at any moment, it has access only to past and present acoustic information. At the same time, it processes sounds across multiple temporal scales, although the computational advantage of this organization remains unclear. Despite both being defining characteristics of biological auditory processing, causal processing and multiscale temporal processing are rarely considered together in computational models. Here, we introduce the Multiscale Convolutional Recurrent Neural Network (MSCRNN) architecture, a brain-inspired model designed to test whether processing sounds across multiple temporal contexts improves recognition under causal constraints. Multiscale processing substantially improved recognition under causal constraints, enabling performance comparable to non-causal architectures after an initial evidence-accumulation period while providing only limited benefit when future acoustic information was available. Analyses of the network’s internal representations revealed a progressive transformation from stream-specific acoustic representations to increasingly integrated category-level representations across recurrent processing stages. Together, these findings provide a computational rationale for why biological auditory systems may benefit from multiscale temporal processing under causal constraints and establish the MSCRNN as an interpretable framework for generating and testing hypotheses about the temporal dynamics of auditory processing using electrophysiological and neuroimaging data.
Open AccessMultiscale Temporal Processing Supports Sound Recognition under Causal Constraints
The auditory system operates under a fundamental computational constraint: at any moment, it has access only to past and present acoustic information. At the same time, it processes sounds across multiple temporal scales, although the computational advantage of this organization remains unclear. Despite both being defining characteristics of biological auditory processing, causal processing and multiscale temporal processing are rarely considered together in computational models. Here, we introduce the Multiscale Convolutional Recurrent Neural Network (MSCRNN) architecture, a brain-inspired model designed to test whether processing sounds across multiple temporal contexts improves recognition under causal constraints. Multiscale processing substantially improved recognition under causal constraints, enabling performance comparable to non-causal architectures after an initial evidence-accumulation period while providing only limited benefit when future acoustic information was available. Analyses of the network’s internal representations revealed a progressive transformation from stream-specific acoustic representations to increasingly integrated category-level representations across recurrent processing stages. Together, these findings provide a computational rationale for why biological auditory systems may benefit from multiscale temporal processing under causal constraints and establish the MSCRNN as an interpretable framework for generating and testing hypotheses about the temporal dynamics of auditory processing using electrophysiological and neuroimaging data.
- Open Access
- 14 April 2026
- Published by eLife Sciences Publications, Ltd in eLife preprints
- Vol. 15
Speech is defined by theta-gamma coupled acoustic rhythms, mapped onto segregated populations in human early auditory cortex
Theta and gamma neural dynamics dominate the human auditory cortex during speech perception and have been proposed to track syllable boundaries and encode phonemic information, respectively. To what extent these rhythms are intrinsically generated or imposed by speech acoustics remains unsolved. Applying analytic methods from neuroscience to speech audio corpora from 17 languages, we found that canonical brain features —theta, gamma, and their phase-amplitude coupling— are a robust and specific acoustic signature of speech envelope across languages. They represent syllabic rate (2–6 Hz), vocalic features (30–50 Hz), and fundamental frequency (100–150 Hz). Intracerebral (sEEG) recordings from the auditory cortex of 18 epilepsy patients revealed that theta-gamma dynamics and their coupling are absent at rest. They emerge during speech perception and are linearly driven by the acoustic envelope, consistent with an evoked origin. Nevertheless, these responses originate from distinct yet functionally interconnected neural populations, indicating that the early auditory cortex demultiplexes speech timescales. Thus, early auditory cortex mirrors theta–gamma speech rhythms across segregated neural populations.
- 14 April 2026
- Published by eLife Sciences Publications, Ltd in eLife preprints
- Vol. 15
Open AccessSpeech is defined by theta-gamma coupled acoustic rhythms, mapped onto segregated populations in human early auditory cortex
Theta and gamma neural dynamics dominate the human auditory cortex during speech perception and have been proposed to track syllable boundaries and encode phonemic information, respectively. To what extent these rhythms are intrinsically generated or imposed by speech acoustics remains unsolved. Applying analytic methods from neuroscience to speech audio corpora from 17 languages, we found that canonical brain features —theta, gamma, and their phase-amplitude coupling— are a robust and specific acoustic signature of speech envelope across languages. They represent syllabic rate (2–6 Hz), vocalic features (30–50 Hz), and fundamental frequency (100–150 Hz). Intracerebral (sEEG) recordings from the auditory cortex of 18 epilepsy patients revealed that theta-gamma dynamics and their coupling are absent at rest. They emerge during speech perception and are linearly driven by the acoustic envelope, consistent with an evoked origin. Nevertheless, these responses originate from distinct yet functionally interconnected neural populations, indicating that the early auditory cortex demultiplexes speech timescales. Thus, early auditory cortex mirrors theta–gamma speech rhythms across segregated neural populations.
- Open Access
- 8 January 2026
- Published by eLife Sciences Publications, Ltd in eLife
- Vol. 13
Reconstructing voice identity from noninvasive auditory cortex recordings
The cerebral processing of voice information is known to engage, in human as well as non-human primates, ‘temporal voice areas’ (TVAs) that respond preferentially to conspecific vocalizations. However, how voice information is represented by neuronal populations in these areas, particularly speaker identity information, remains poorly understood. Here, we used a deep neural network (DNN) to generate a high-level, small-dimension representational space for voice identity—the ‘voice latent space’ (VLS)—and examined its linear relation with cerebral activity via encoding, representational similarity, and decoding analyses. We find that the VLS maps onto fMRI measures of cerebral activity in response to tens of thousands of voice stimuli from hundreds of different speaker identities and better accounts for the representational geometry for speaker identity in the TVAs than in A1. Moreover, the VLS allowed TVA-based reconstructions of voice stimuli that preserved essential aspects of speaker identity as assessed by both machine classifiers and human listeners. These results indicate that the DNN-derived VLS provides high-level representations of voice identity information in the TVAs.
- 8 January 2026
- Published by eLife Sciences Publications, Ltd in eLife
- Vol. 13
Open AccessReconstructing voice identity from noninvasive auditory cortex recordings
The cerebral processing of voice information is known to engage, in human as well as non-human primates, ‘temporal voice areas’ (TVAs) that respond preferentially to conspecific vocalizations. However, how voice information is represented by neuronal populations in these areas, particularly speaker identity information, remains poorly understood. Here, we used a deep neural network (DNN) to generate a high-level, small-dimension representational space for voice identity—the ‘voice latent space’ (VLS)—and examined its linear relation with cerebral activity via encoding, representational similarity, and decoding analyses. We find that the VLS maps onto fMRI measures of cerebral activity in response to tens of thousands of voice stimuli from hundreds of different speaker identities and better accounts for the representational geometry for speaker identity in the TVAs than in A1. Moreover, the VLS allowed TVA-based reconstructions of voice stimuli that preserved essential aspects of speaker identity as assessed by both machine classifiers and human listeners. These results indicate that the DNN-derived VLS provides high-level representations of voice identity information in the TVAs.
- Open Access
- 20 November 2025
- Published by eLife Sciences Publications, Ltd in eLife preprints
Reconstructing Voice Identity from Noninvasive Auditory Cortex Recordings
The cerebral processing of voice information is known to engage, in human as well as non-human primates, “temporal voice areas” (TVAs) that respond preferentially to conspecific vocalizations. However, how voice information is represented by neuronal populations in these areas, particularly speaker identity information, remains poorly understood. Here, we used a deep neural network (DNN) to generate a high-level, small-dimension representational space for voice identity—the ‘voice latent space’ (VLS)—and examined its linear relation with cerebral activity via encoding, representational similarity, and decoding analyses. We find that the VLS maps onto fMRI measures of cerebral activity in response to tens of thousands of voice stimuli from hundreds of different speaker identities and better accounts for the representational geometry for speaker identity in the TVAs than in A1. Moreover, the VLS allowed TVA-based reconstructions of voice stimuli that preserved essential aspects of speaker identity as assessed by both machine classifiers and human listeners. These results indicate that the DNN-derived VLS provides high-level representations of voice identity information in the TVAs.
- 20 November 2025
- Published by eLife Sciences Publications, Ltd in eLife preprints
Open AccessReconstructing Voice Identity from Noninvasive Auditory Cortex Recordings
The cerebral processing of voice information is known to engage, in human as well as non-human primates, “temporal voice areas” (TVAs) that respond preferentially to conspecific vocalizations. However, how voice information is represented by neuronal populations in these areas, particularly speaker identity information, remains poorly understood. Here, we used a deep neural network (DNN) to generate a high-level, small-dimension representational space for voice identity—the ‘voice latent space’ (VLS)—and examined its linear relation with cerebral activity via encoding, representational similarity, and decoding analyses. We find that the VLS maps onto fMRI measures of cerebral activity in response to tens of thousands of voice stimuli from hundreds of different speaker identities and better accounts for the representational geometry for speaker identity in the TVAs than in A1. Moreover, the VLS allowed TVA-based reconstructions of voice stimuli that preserved essential aspects of speaker identity as assessed by both machine classifiers and human listeners. These results indicate that the DNN-derived VLS provides high-level representations of voice identity information in the TVAs.
- Open Access
Speech is defined by theta-gamma coupled acoustic rhythms, mapped onto segregated populations in human early auditory cortex
Theta and gamma neural dynamics dominate the human auditory cortex during speech perception and have been proposed to track syllable boundaries and encode phonemic information, respectively. To what extent these rhythms are intrinsically generated or imposed by speech acoustics remains unsolved. Applying analytic methods from neuroscience to speech audio corpora from 17 languages, we found that canonical brain features —theta, gamma, and their phase-amplitude coupling— are a robust and specific acoustic signature of speech envelope across languages. They represent syllabic rate (2–6 Hz), vocalic features (30–50 Hz), and fundamental frequency (100–150 Hz). Intracerebral (sEEG) recordings from the auditory cortex of 18 epilepsy patients revealed that theta-gamma dynamics and their coupling are absent at rest. They emerge during speech perception and are linearly driven by the acoustic envelope, consistent with an evoked origin. Nevertheless, these responses originate from distinct yet functionally interconnected neural populations, indicating that the early auditory cortex demultiplexes speech timescales. Thus, early auditory cortex mirrors theta–gamma speech rhythms across segregated neural populations. IN BRIEF: Early auditory cortex demultiplexes speech: stimulus-locked gamma activity aligns theta phase with the nested acoustic structure, functionally organising speech timescales
Open AccessSpeech is defined by theta-gamma coupled acoustic rhythms, mapped onto segregated populations in human early auditory cortex
Theta and gamma neural dynamics dominate the human auditory cortex during speech perception and have been proposed to track syllable boundaries and encode phonemic information, respectively. To what extent these rhythms are intrinsically generated or imposed by speech acoustics remains unsolved. Applying analytic methods from neuroscience to speech audio corpora from 17 languages, we found that canonical brain features —theta, gamma, and their phase-amplitude coupling— are a robust and specific acoustic signature of speech envelope across languages. They represent syllabic rate (2–6 Hz), vocalic features (30–50 Hz), and fundamental frequency (100–150 Hz). Intracerebral (sEEG) recordings from the auditory cortex of 18 epilepsy patients revealed that theta-gamma dynamics and their coupling are absent at rest. They emerge during speech perception and are linearly driven by the acoustic envelope, consistent with an evoked origin. Nevertheless, these responses originate from distinct yet functionally interconnected neural populations, indicating that the early auditory cortex demultiplexes speech timescales. Thus, early auditory cortex mirrors theta–gamma speech rhythms across segregated neural populations. IN BRIEF: Early auditory cortex demultiplexes speech: stimulus-locked gamma activity aligns theta phase with the nested acoustic structure, functionally organising speech timescales
- Open Access
Bridging Auditory Perception and Natural Language Processing with Semantically informed Deep Neural Networks
Sound recognition is effortless for humans but poses a significant chal-lenge for artificial hearing systems. Deep neural networks (DNNs), especially convolutional neural networks (CNNs), have recently sur-passed traditional machine learning in sound classification. However, current DNNs map sounds to labels using binary categorical variables, neglecting the semantic relations between labels. Cognitive neuroscience research suggests that human listeners exploit such semantic informa-tion besides acoustic cues. Hence, our hypothesis is that incorporating semantic information improves DNN’s sound recognition performance, emulating human behavior. In our approach, sound recognition is framed as a regression problem, with CNNs trained to map spec-trograms to continuous semantic representations from NLP models (Word2Vec, BERT, and CLAP text encoder). Two DNN types were trained: semDNN with continuous embeddings and catDNN with cat-egorical labels, both with a dataset extracted from a collection of 388,211 sounds enriched with semantic descriptions. Evaluations across four external datasets, confirmed the superiority of semantic labeling from semDNN compared to catDNN, preserving higher-level relations. Importantly, an analysis of human similarity ratings for natural sounds, showed that semDNN approximated human listener behavior better than catDNN, other DNNs, and NLP models. Our work contributes to understanding the role of semantics in sound recognition, bridging the gap between artificial systems and human auditory perception.
Open AccessBridging Auditory Perception and Natural Language Processing with Semantically informed Deep Neural Networks
Sound recognition is effortless for humans but poses a significant chal-lenge for artificial hearing systems. Deep neural networks (DNNs), especially convolutional neural networks (CNNs), have recently sur-passed traditional machine learning in sound classification. However, current DNNs map sounds to labels using binary categorical variables, neglecting the semantic relations between labels. Cognitive neuroscience research suggests that human listeners exploit such semantic informa-tion besides acoustic cues. Hence, our hypothesis is that incorporating semantic information improves DNN’s sound recognition performance, emulating human behavior. In our approach, sound recognition is framed as a regression problem, with CNNs trained to map spec-trograms to continuous semantic representations from NLP models (Word2Vec, BERT, and CLAP text encoder). Two DNN types were trained: semDNN with continuous embeddings and catDNN with cat-egorical labels, both with a dataset extracted from a collection of 388,211 sounds enriched with semantic descriptions. Evaluations across four external datasets, confirmed the superiority of semantic labeling from semDNN compared to catDNN, preserving higher-level relations. Importantly, an analysis of human similarity ratings for natural sounds, showed that semDNN approximated human listener behavior better than catDNN, other DNNs, and NLP models. Our work contributes to understanding the role of semantics in sound recognition, bridging the gap between artificial systems and human auditory perception.
- Open Access
Reconstructing Voice Identity from Noninvasive Auditory Cortex Recordings
The cerebral processing of voice information is known to engage, in human as well as non-human primates, “temporal voice areas” (TVAs) that respond preferentially to conspecific vocalizations. However, how voice information is represented by neuronal populations in these areas, particularly speaker identity information, remains poorly understood. Here, we used a deep neural network (DNN) to generate a high-level, small-dimension representational space for voice identity—the ‘voice latent space’ (VLS)—and examined its linear relation with cerebral activity via encoding, representational similarity, and decoding analyses. We find that the VLS maps onto fMRI measures of cerebral activity in response to tens of thousands of voice stimuli from hundreds of different speaker identities and better accounts for the representational geometry for speaker identity in the TVAs than in A1. Moreover, the VLS allowed TVA-based reconstructions of voice stimuli that preserved essential aspects of speaker identity as assessed by both machine classifiers and human listeners. These results indicate that the DNN-derived VLS provides high-level representations of voice identity information in the TVAs.
Open AccessReconstructing Voice Identity from Noninvasive Auditory Cortex Recordings
The cerebral processing of voice information is known to engage, in human as well as non-human primates, “temporal voice areas” (TVAs) that respond preferentially to conspecific vocalizations. However, how voice information is represented by neuronal populations in these areas, particularly speaker identity information, remains poorly understood. Here, we used a deep neural network (DNN) to generate a high-level, small-dimension representational space for voice identity—the ‘voice latent space’ (VLS)—and examined its linear relation with cerebral activity via encoding, representational similarity, and decoding analyses. We find that the VLS maps onto fMRI measures of cerebral activity in response to tens of thousands of voice stimuli from hundreds of different speaker identities and better accounts for the representational geometry for speaker identity in the TVAs than in A1. Moreover, the VLS allowed TVA-based reconstructions of voice stimuli that preserved essential aspects of speaker identity as assessed by both machine classifiers and human listeners. These results indicate that the DNN-derived VLS provides high-level representations of voice identity information in the TVAs.
- Open Access
- 4 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 770-774
ACES: Evaluating Automated Audio Captioning Models on the Semantics of Sounds
Automated Audio Captioning is a multimodal task that aims to convert audio content into natural language. The performance of audio captioning systems is evaluated on quantitative metrics applied to the text representations. Previously, researchers have applied metrics from machine translation and image captioning to evaluate a generated audio caption. Inspired by cognitive neuroscience research on auditory cognition, in this paper we present a novel metric approach that evaluates captions taking into account how human listeners derive semantic information from sounds: Audio Captioning Evaluation on Semantics of Sound (ACES).
- 4 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 770-774
Open AccessACES: Evaluating Automated Audio Captioning Models on the Semantics of Sounds
Automated Audio Captioning is a multimodal task that aims to convert audio content into natural language. The performance of audio captioning systems is evaluated on quantitative metrics applied to the text representations. Previously, researchers have applied metrics from machine translation and image captioning to evaluate a generated audio caption. Inspired by cognitive neuroscience research on auditory cognition, in this paper we present a novel metric approach that evaluates captions taking into account how human listeners derive semantic information from sounds: Audio Captioning Evaluation on Semantics of Sound (ACES).
- Top Cited PapersOpen Access
- 16 March 2023
- Published by Springer Nature in Nature Neuroscience
- Vol. 26 (4) , 664-672
Intermediate acoustic-to-semantic representations link behavioral and neural responses to natural sounds
Recognizing sounds implicates the cerebral transformation of input waveforms into semantic representations. Although past research identified the superior temporal gyrus (STG) as a crucial cortical region, the computational fingerprint of these cerebral transformations remains poorly characterized. Here, we exploit a model comparison framework and contrasted the ability of acoustic, semantic (continuous and categorical) and sound-to-event deep neural network representation models to predict perceived sound dissimilarity and 7 T human auditory cortex functional magnetic resonance imaging responses. We confirm that spectrotemporal modulations predict early auditory cortex (Heschl’s gyrus) responses, and that auditory dimensions (for example, loudness, periodicity) predict STG responses and perceived dissimilarity. Sound-to-event deep neural networks predict Heschl’s gyrus responses similar to acoustic models but, notably, they outperform all competing models at predicting both STG responses and perceived dissimilarity. Our findings indicate that STG entails intermediate acoustic-to-semantic sound representations that neither acoustic nor semantic models can account for. These representations are compositional in nature and relevant to behavior.
- 16 March 2023
- Published by Springer Nature in Nature Neuroscience
- Vol. 26 (4) , 664-672
Top Cited PapersOpen AccessIntermediate acoustic-to-semantic representations link behavioral and neural responses to natural sounds
Recognizing sounds implicates the cerebral transformation of input waveforms into semantic representations. Although past research identified the superior temporal gyrus (STG) as a crucial cortical region, the computational fingerprint of these cerebral transformations remains poorly characterized. Here, we exploit a model comparison framework and contrasted the ability of acoustic, semantic (continuous and categorical) and sound-to-event deep neural network representation models to predict perceived sound dissimilarity and 7 T human auditory cortex functional magnetic resonance imaging responses. We confirm that spectrotemporal modulations predict early auditory cortex (Heschl’s gyrus) responses, and that auditory dimensions (for example, loudness, periodicity) predict STG responses and perceived dissimilarity. Sound-to-event deep neural networks predict Heschl’s gyrus responses similar to acoustic models but, notably, they outperform all competing models at predicting both STG responses and perceived dissimilarity. Our findings indicate that STG entails intermediate acoustic-to-semantic sound representations that neither acoustic nor semantic models can account for. These representations are compositional in nature and relevant to behavior.