Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Thierry Chaminade has a total of 48 co-authors. Here are the top co-authors by number of shared publications:

  • Farah H. Wolfe

    A Institut de Neurosciences de la Timone, UMR7289 , Aix-Marseille Université, CNRS , Marseille , France

    3 shared publications
  • Laurent Prévot

    LPL UMR 7309, CNRS, Aix Marseille Université, Marseille, France

    3 shared publications
  • Magalie Ochs

    LIS UMR 7020, CNRS, Aix Marseille Université, Université de Toulon, Marseille, France

    3 shared publications
  • Nicolas Spatola

    Artimon Perspectives, Paris, France

    3 shared publications
  • Antonin Marty

    Institut de Neurosciences de la Timone UMR 7289 Aix-Marseille Université - CNRS, Marseille cedex 5, France

    2 shared publications
  • Arthur Pineaud

    Institut de Neurosciences de la Timone UMR 7289 Aix-Marseille Université - CNRS, Marseille cedex 5, France

    2 shared publications

Latest Publications

  • Open Access

    Central nervous system control of breathing in natural conversation turn-taking

    by Camilla Di Pasquasio, Lila De Pellegrin, Arthur Pineaud, Antonin Marty, Thierry Chaminade

    Conversations—hallmarks of human social interaction—rely on a deeply rooted evolutionary tool: vocalization. Vocalization productions begin with an airflow from the lungs, shaped by the vocal folds and articulators of the mouth to generate a rich variety of sounds. Speech production therefore involves tight synchronisation between respiratory and articulatory events, especially in natural social interactions. During conversations, turn-taking indeed implies respiration anticipation: lungs must be filled with enough air to produce the next speech segment. Here, we use an fMRI corpus of live, unconstrained conversations to identify key brain regions involved in the respiratory anticipation of speech onset. Behaviourally, respiratory local maxima preceded speech onset (“Resp+” events) by approximately 200 ms, consistent with volitional control of breathing overriding automatic respiratory rhythms. Contrasting Resp+ events with respiratory maxima not linked to speech onsets (“Resp−” events) revealed significant (p FWE < 0.05, k > 5 cm³) bilateral responses in the postcentral sulcus, brainstem, and cerebellum. Brainstem respiratory control nuclei are known to generate automatic breathing rhythms while integrating multimodal signals for gas exchange homeostasis. The postcentral sulcus has been implicated in processing afferent feedback from respiratory effectors during intentional breathing, and the cerebellum maintains reciprocal anatomical connections with both brainstem respiratory centres and cortical respiratory control areas. Together, these structures form a network that inhibits automatic respiration for volitional respiratory control of speech onset. Our findings underscore the interaction between cortical and subcortical regions involved in respiratory control for speech production in social interactions, and offer new insights into the neural foundations of human communication and its disorders.


  • Open Access

    Predicting Activity in Brain Areas Associated with Emotion Processing Using Multimodal Behavioral Signals

    by Lahoucine Kdouri, Youssef Hmamouche, Amal El Fallah Seghrouchni, Thierry Chaminade

    Artificial agents are expected to increasingly interact with humans and to demonstrate multimodal adaptive emotional responses. Such social integration requires both perception and production mechanisms, thus enabling a more realistic approach to emotional alignment than existing systems. Indeed, existing emotion recognition methods rely on behavioral signals, predominantly facial expressions, as well as non-invasive brain recordings, such as Electroencephalograms (EEGs) and functional Magnetic Resonance Imaging (fMRI), to identify humans’ emotions, but accurate labeling remains a challenge. This paper introduces a novel approach examining how behavioral and physiological signals can be used to predict activity in emotion-related regions of the brain. To this end, we propose a multimodal deep learning network that processes two categories of signals recorded alongside brain activity during conversations: two behavioral signals (video and audio) and one physiological signal (blood pulse). Our network enables (1) the prediction of brain activity from these multimodal inputs, and (2) the assessment of our model’s performance depending on the nature of interlocutor (human or robot) and the brain region of interest. Results demonstrate that the proposed architecture outperforms existing models in anterior insula and hypothalamus regions, for interactions with a human or a robot. An ablation study evaluating subsets of input modalities indicates that local brain activity prediction was reduced when one or two modalities are omitted. However, they also revealed that the physiological data (blood pulse) achieve similar levels of predictions alone compared to the full model, further underscoring the importance of somatic markers in the central nervous system’s processing of social emotions.


  • Open Access

    Central Nervous System Control of breathing in Natural Conversational Turn-Taking

    by Camilla Di Pasquasio, Lila De Pellegrin, Arthur Pineaud, Antonin Marty, Thierry Chaminade

    Discussion is a fundamental social activity requiring coordination of speech between interlocutors. Speech production is a complex human behaviour that involves several anatomo-physiological processes, including inspiration and expiration. The aim of the present study is to investigate the neurophysiological underpinnings of speech-related respiration events in conversational turn-taking. We made use of an existing corpus of natural conversations between a participant and its interlocutor (Human or Robot) focusing on synchronised (1) behavioural (conversation turn-taking), (2) respiratory (maxima of inspiration) and (3) neurophysiological (fMRI) data. Precisely timed conversation transcripts from 25 participants were used to categorise breathing maxima based on their timing relative to the participant’s speech onset. In agreement with the literature, the closest respiration time maximum to each speech turn occurred on average 200 ms prior to speech onset. The fMRI second-level contrast ( p FWE < 0.05, extend k > 5 cm 3 ) Resp+ (maximum respiration associated with speech) versus Resp-, exclusively masked to exclude speech related areas, revealed bilateral activations in the central sulcus, the brainstem and the cerebellum. The brainstem cluster comprises respiratory pattern generators, possibly the preBötzinger complex, that need to be inhibited to enslave breathing to speech production and not physiological needs, while the central sulcus cluster is likely to be located in the postcentral primary sensory cortex receiving upper torso inputs indicating that lungs are filled, and the cerebellum clusters could play a role in the timing of speech onset, 200 ms after a respiration maximum. These results show how cortical, cerebellar and brainstem coordinated control of breathing during conversational turn-taking is part of the intricate physiological mechanisms that contribute to natural communication dynamics.

  • Open Access

    Interpretable prediction of brain activity during conversations from multimodal behavioral signals

    by Youssef Hmamouche, Magalie Ochs, Laurent Prévot, Thierry Chaminade

    We present an analytical framework aimed at predicting the local brain activity in uncontrolled experimental conditions based on multimodal recordings of participants’ behavior, and its application to a corpus of participants having conversations with another human or a conversational humanoid robot. The framework consists in extracting high-level features from the raw behavioral recordings and applying a dynamic prediction of binarized fMRI-recorded local brain activity using these behavioral features. The objective is to identify behavioral features required for this prediction, and their relative weights, depending on the brain area under investigation and the experimental condition. In order to validate our framework, we use a corpus of uncontrolled conversations of participants with a human or a robotic agent, focusing on brain regions involved in speech processing, and more generally in social interactions. The framework not only predicts local brain activity significantly better than random, it also quantifies the weights of behavioral features required for this prediction, depending on the brain area under investigation and on the nature of the conversational partner. In the left Superior Temporal Sulcus, perceived speech is the most important behavioral feature for predicting brain activity, regardless of the agent, while several features, which differ between the human and robot interlocutors, contribute to the prediction in regions involved in social cognition, such as the TemporoParietal Junction. This framework therefore allows us to study how multiple behavioral signals from different modalities are integrated in individual brain regions during complex social interactions.


    • 29 November 2023
    • Published by HAL in HAL
    Open Access

    Interpretable Prediction of Brain Activity during Natural Social Interactions using Multimodal Behavioral Signals

    by Youssef Hmamouche, Magalie Ochs, Laurent Prevot, Thierry Chaminade, Laurent Prévot

    We present an analytical framework aimed at predicting the local brain activity of individuals during a conversation with another human or a humanoid robot based on multimodal recordings of their behavior. In this framework, we first extract high-level features from the raw behavioral recordings of both interlocutors. Then, classifiers are applied to predict binarized brain activity from these features using a dynamic prediction model. Here, we focus on brain regions involved in social interactions, both speech processing involved areas in conversations and information integration areas, in order to validate our framework. This framework not only predicts local brain activity significantly better than random, but it also identifies the behavioral features required for this prediction depending on the brain area under investigation and on the nature of the conversational partner. In the left Superior Temporal Sulcus, perceived speech is the most important behavioral feature for predicting brain activity, regardless of the agent, while multiple features, which differ between the human and robot interlocutors, contribute to prediction in regions involved in social signal integration, such as the TemporoParietal Junction. This framework allows us to study how multiple behavioral signals from different modalities are integrated in individual brain regions during inherently complex unconstrained natural social interactions.

  • Open Access

    Social Context and Drug Cues Modulate Inhibitory Control in Cocaine Addiction: involvement of the STN evidenced through Functional Magnetic Resonance Imaging

    by Damiano Terenzi, Nicolas Simon, Michael Joe Munyua Gachomba, Jeanne Laure de Peretti, Bruno Nazarian, Julien Sein, Jean-Luc Anton, Didier Grandjean, Christelle Baunez, Thierry Chaminade

    Background Addictions often develop in a social context, although the influence of social factors did not receive much attention in the neuroscience of addiction. Recent animal studies suggest that peer presence can reduce cocaine intake, an influence potentially mediated, among others, by the subthalamic nucleus (STN). However, there is to date no such neurobiological study in humans. Methods This study investigated the impact of social context and drug cues on brain correlates of inhibitory control in individuals with and without cocaine use disorder (CUD) using functional Magnetic Resonance Imaging (fMRI). Seventeen CUD participants and 17 healthy controls (HC) performed a novel fMRI Stop-Signal task in the presence and absence of an observer while being exposed to cocaine-related and neutral pictures as cues. Results The results showed that CUD participants, while slower at stopping with neutral cues, recovered control level stopping abilities with cocaine cues, while HC did not show any difference. Neuroimaging revealed the involvement of frontal cortical regions and of the subthalamic nucleus (STN) in inhibitory control, which was modulated by both social context and drug cues in CUD participants but not in HC. Conclusions These findings highlight the impact of social context and drug cues on inhibitory control in CUD and suggest potential targets for intervention such as the STN, also emphasizing the importance of considering the social context in addiction research and treatment.


  • Open Access

    Perceived facial happiness during conversation correlates with insular and hypothalamus activity for humans, not robots

    by Thierry Chaminade, Nicolas Spatola

    Emotional contagion, in particular of happiness, is essential to creating social bonds. The somatic marker hypothesis posits that embodied physiological changes associated with emotions and relayed to the brain by the autonomous nervous system influence behavior. Perceiving others’ positive emotions should thus be associated with activity in brain regions relaying information from and to the autonomic nervous system. Here, we address this question using a unique corpus of brain activity recorded during unconstrained conversations between participants and a human or a humanoid robot. fMRI recordings are used to test whether activity in key brain regions of the autonomic system, the amygdala, hypothalamus, and insula, is differentially affected by the level of happiness expressed by the human and robot agents. Results indicate that for the hypothalamus and the insula, in particular the anterior agranular region strongly involved in processing social emotions, activity in the right hemisphere increases with the level of happiness expressed by the human but not the robot. Perceiving positive emotions in social interactions induces local brain responses predicted by the contagion of somatic markers of emotions only when the interacting agent is a fellow human.


  • Open Access

    Precuneus brain response changes differently during human–robot and human–human dyadic social interaction

    by Nicolas Spatola, Thierry Chaminade

    Human–human interactions (HHI) and human–robot interactions (HRI) are compared to identify differences between cognitive processes reflecting bonding in social interactions with natural and artificial agents. We capitalize on a unique corpus of neuroimaging data (fMRI) recorded while participants freely discussed with another human or a conversational robotic head, in order to study a crucial parameter of human social cognition, namely that social interactions are adaptive bidirectional processes that evolve over time. We used linear statistics to identify regions of the brain where activity changes differently when participants carry out twelve one-minute conversations, alternating between a human and a robotic interlocutor. Results show that activity in the posterior cingulate cortex, a key region associated with social cognition, increases over time in HHI but not in HRI. These results are interpreted as reflecting a process of strengthening social bonding during repeated exchanges when the interacting agent is a human, but not a robot.


  • Open Access

    Causal Analysis of Activity in Social Brain Areas During Human-Agent Conversation

    by Caio De Castro Martins, Thierry Chaminade, Marc Cavazza

    This article investigates the differences in cognitive and neural mechanisms between human-human and human-virtual agent interaction using a dataset recorded in an ecologically realistic environment. We use Convergent Cross Mapping (CCM) to investigate functional connectivity between pairs of regions involved in the framework of social cognitive neuroscience, namely the fusiform gyrus, superior temporal sulcus (STS), temporoparietal junction (TPJ), and the dorsolateral prefrontal cortex (DLPFC)—taken as prefrontal asymmetry. Our approach is a compromise between investigating local activation in specific regions and investigating connectivity networks that may form part of larger networks. In addition to concording with previous studies, our results suggest that the right TPJ is one of the most reliable areas for assessing processes occurring during human-virtual agent interactions, both in a static and dynamic sense.


  • Open Access

    Perceived facial happiness during conversation correlates with insular and hypothalamus activity for humans, not robots: An investigation of the somatic marker hypothesis applied to social interactions

    by Thierry Chaminade, Nicolas Spatola

    The somatic marker hypothesis posits that perceiving emotions entails reenacting markers of self emotions, in particular in the autonomous nervous system. Well studied in decision-making tasks, it has not been tested in a social cognitive neuroscience framework, and in particular for the automatic processing of positive emotions during natural interactions. Here, we address this question using a unique corpus of brain activity recorded during unconstrained conversations between participants and a human or a humanoid robot. fMRI recordings are used to test whether activity in the most important brain regions in relation to the autonomic system, the amygdala, hypothalamus and insula, is affected by the level of happiness expressed by the human and robot agents. Results indicate that for the hypothalamus and the insula, in particular the anterior agranular region strongly involved in processing social emotions, activity in the right hemisphere increases with the level of happiness expressed by the human, but not the robot. Results indicate that perceiving positive emotions in social interactions induces the local brain responses predicted by the somatic marker hypothesis, but only when the interacting agent is a real human.