Stéphane Ayache
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Stéphane Ayache has a total of 34 co-authors. Here are the top co-authors by number of shared publications:
- Thierry Artières
Laboratoire d’Informatique et Systèmes UMR 7020, CNRS, Aix-Marseille University
4 shared publications - Alice Delbosc
Davi, The Humanizers, Puteaux, France
3 shared publications - Brian Ravenet
University Paris-Saclay, Orsay, France
3 shared publications - Bruno L. Giordano
La Timone Neuroscience Institute UMR 7289, CNRS, Aix-Marseille University
3 shared publications - Charly Lamothe
Institut Pasteur, AP-HP, INSERM, CNRS, Fondation Pour l'Audition, Institut de l'Audition, IHU reConnect, Université Paris Cité, Paris, France
3 shared publications - Etienne Thoret
Aix-Marseille University, CNRS, INT, UMR7289 1 , F-13005 Marseille
3 shared publications
Latest Publications
- Open Access
- 8 January 2026
- Published by eLife Sciences Publications, Ltd in eLife
- Vol. 13
Reconstructing voice identity from noninvasive auditory cortex recordings
The cerebral processing of voice information is known to engage, in human as well as non-human primates, ‘temporal voice areas’ (TVAs) that respond preferentially to conspecific vocalizations. However, how voice information is represented by neuronal populations in these areas, particularly speaker identity information, remains poorly understood. Here, we used a deep neural network (DNN) to generate a high-level, small-dimension representational space for voice identity—the ‘voice latent space’ (VLS)—and examined its linear relation with cerebral activity via encoding, representational similarity, and decoding analyses. We find that the VLS maps onto fMRI measures of cerebral activity in response to tens of thousands of voice stimuli from hundreds of different speaker identities and better accounts for the representational geometry for speaker identity in the TVAs than in A1. Moreover, the VLS allowed TVA-based reconstructions of voice stimuli that preserved essential aspects of speaker identity as assessed by both machine classifiers and human listeners. These results indicate that the DNN-derived VLS provides high-level representations of voice identity information in the TVAs.
- 8 January 2026
- Published by eLife Sciences Publications, Ltd in eLife
- Vol. 13
Open AccessReconstructing voice identity from noninvasive auditory cortex recordings
The cerebral processing of voice information is known to engage, in human as well as non-human primates, ‘temporal voice areas’ (TVAs) that respond preferentially to conspecific vocalizations. However, how voice information is represented by neuronal populations in these areas, particularly speaker identity information, remains poorly understood. Here, we used a deep neural network (DNN) to generate a high-level, small-dimension representational space for voice identity—the ‘voice latent space’ (VLS)—and examined its linear relation with cerebral activity via encoding, representational similarity, and decoding analyses. We find that the VLS maps onto fMRI measures of cerebral activity in response to tens of thousands of voice stimuli from hundreds of different speaker identities and better accounts for the representational geometry for speaker identity in the TVAs than in A1. Moreover, the VLS allowed TVA-based reconstructions of voice stimuli that preserved essential aspects of speaker identity as assessed by both machine classifiers and human listeners. These results indicate that the DNN-derived VLS provides high-level representations of voice identity information in the TVAs.
- Open Access
- 20 November 2025
- Published by eLife Sciences Publications, Ltd in eLife preprints
Reconstructing Voice Identity from Noninvasive Auditory Cortex Recordings
The cerebral processing of voice information is known to engage, in human as well as non-human primates, “temporal voice areas” (TVAs) that respond preferentially to conspecific vocalizations. However, how voice information is represented by neuronal populations in these areas, particularly speaker identity information, remains poorly understood. Here, we used a deep neural network (DNN) to generate a high-level, small-dimension representational space for voice identity—the ‘voice latent space’ (VLS)—and examined its linear relation with cerebral activity via encoding, representational similarity, and decoding analyses. We find that the VLS maps onto fMRI measures of cerebral activity in response to tens of thousands of voice stimuli from hundreds of different speaker identities and better accounts for the representational geometry for speaker identity in the TVAs than in A1. Moreover, the VLS allowed TVA-based reconstructions of voice stimuli that preserved essential aspects of speaker identity as assessed by both machine classifiers and human listeners. These results indicate that the DNN-derived VLS provides high-level representations of voice identity information in the TVAs.
- 20 November 2025
- Published by eLife Sciences Publications, Ltd in eLife preprints
Open AccessReconstructing Voice Identity from Noninvasive Auditory Cortex Recordings
The cerebral processing of voice information is known to engage, in human as well as non-human primates, “temporal voice areas” (TVAs) that respond preferentially to conspecific vocalizations. However, how voice information is represented by neuronal populations in these areas, particularly speaker identity information, remains poorly understood. Here, we used a deep neural network (DNN) to generate a high-level, small-dimension representational space for voice identity—the ‘voice latent space’ (VLS)—and examined its linear relation with cerebral activity via encoding, representational similarity, and decoding analyses. We find that the VLS maps onto fMRI measures of cerebral activity in response to tens of thousands of voice stimuli from hundreds of different speaker identities and better accounts for the representational geometry for speaker identity in the TVAs than in A1. Moreover, the VLS allowed TVA-based reconstructions of voice stimuli that preserved essential aspects of speaker identity as assessed by both machine classifiers and human listeners. These results indicate that the DNN-derived VLS provides high-level representations of voice identity information in the TVAs.
- Open Access
- 16 September 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-4
Automatic objective metric for the optimization of nonverbal behavior generative models
- 16 September 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-4
Open AccessAutomatic objective metric for the optimization of nonverbal behavior generative models
- Open Access
AI-Driven Plasma Denaturation Profiling for Multi-Cancer Detection
Many cancers cannot be detected early due to lack of effective disease biomarkers, leading to poor prognosis. We applied an existing biophysical technology nanoDSF in a novel way to answer this unmet biomedical need. We developed a breakthrough digital biomarker method for cancer detection based on AI-classification of plasma denaturation profiles (PDPs) obtained by nanoDSF technology. PDPs from 300 plasma samples from patients with melanoma, brain, digestive or lung cancers were automatically distinguished from ‘healthy’ profiles with an accuracy of 94%. Moreover, our method was able to distinguish different types of cancers from each other with an accuracy of 80%, making it an effective way to help cancer diagnosis and monitoring. Our technology thus paves the way for a long-sought multi-cancer early detection (MCED) test that is blood-based, cost-effective and easy-to-implement in any clinical setting.
Open AccessAI-Driven Plasma Denaturation Profiling for Multi-Cancer Detection
Many cancers cannot be detected early due to lack of effective disease biomarkers, leading to poor prognosis. We applied an existing biophysical technology nanoDSF in a novel way to answer this unmet biomedical need. We developed a breakthrough digital biomarker method for cancer detection based on AI-classification of plasma denaturation profiles (PDPs) obtained by nanoDSF technology. PDPs from 300 plasma samples from patients with melanoma, brain, digestive or lung cancers were automatically distinguished from ‘healthy’ profiles with an accuracy of 94%. Moreover, our method was able to distinguish different types of cancers from each other with an accuracy of 80%, making it an effective way to help cancer diagnosis and monitoring. Our technology thus paves the way for a long-sought multi-cancer early detection (MCED) test that is blood-based, cost-effective and easy-to-implement in any clinical setting.
- Open Access
- 4 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 284-292
Mitigation of gender bias in automatic facial non-verbal behaviors generation
Research on non-verbal behavior generation for social interactive agents focuses mainly on the believability and synchronization of non-verbal cues with speech. However, existing models, predominantly based on deep learning architectures, often perpetuate biases inherent in the training data. This raises ethical concerns, depending on the intended application of these agents. This paper addresses these issues by first examining the influence of gender on facial non-verbal behaviors. We concentrate on gaze, head movements, and facial expressions. We introduce a classifier capable of discerning the gender of a speaker from their non-verbal cues. This classifier achieves high accuracy on both real behavior data, extracted using state-of-the-art tools, and synthetic data, generated from a model developed in previous work. Building upon this work, we present a new model, FairGenderGen, which integrates a gender discriminator and a gradient reversal layer into our previous behavior generation model. This new model generates facial non-verbal behaviors from speech features, mitigating gender sensitivity in the generated behaviors. Our experiments demonstrate that the classifier, developed in the initial phase, is no longer effective in distinguishing the gender of the speaker from the generated non-verbal behaviors.
- 4 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 284-292
Open AccessMitigation of gender bias in automatic facial non-verbal behaviors generation
Research on non-verbal behavior generation for social interactive agents focuses mainly on the believability and synchronization of non-verbal cues with speech. However, existing models, predominantly based on deep learning architectures, often perpetuate biases inherent in the training data. This raises ethical concerns, depending on the intended application of these agents. This paper addresses these issues by first examining the influence of gender on facial non-verbal behaviors. We concentrate on gaze, head movements, and facial expressions. We introduce a classifier capable of discerning the gender of a speaker from their non-verbal cues. This classifier achieves high accuracy on both real behavior data, extracted using state-of-the-art tools, and synthetic data, generated from a model developed in previous work. Building upon this work, we present a new model, FairGenderGen, which integrates a gender discriminator and a gradient reversal layer into our previous behavior generation model. This new model generates facial non-verbal behaviors from speech features, mitigating gender sensitivity in the generated behaviors. Our experiments demonstrate that the classifier, developed in the initial phase, is no longer effective in distinguishing the gender of the speaker from the generated non-verbal behaviors.
- Open Access
Reconstructing Voice Identity from Noninvasive Auditory Cortex Recordings
The cerebral processing of voice information is known to engage, in human as well as non-human primates, “temporal voice areas” (TVAs) that respond preferentially to conspecific vocalizations. However, how voice information is represented by neuronal populations in these areas, particularly speaker identity information, remains poorly understood. Here, we used a deep neural network (DNN) to generate a high-level, small-dimension representational space for voice identity—the ‘voice latent space’ (VLS)—and examined its linear relation with cerebral activity via encoding, representational similarity, and decoding analyses. We find that the VLS maps onto fMRI measures of cerebral activity in response to tens of thousands of voice stimuli from hundreds of different speaker identities and better accounts for the representational geometry for speaker identity in the TVAs than in A1. Moreover, the VLS allowed TVA-based reconstructions of voice stimuli that preserved essential aspects of speaker identity as assessed by both machine classifiers and human listeners. These results indicate that the DNN-derived VLS provides high-level representations of voice identity information in the TVAs.
Open AccessReconstructing Voice Identity from Noninvasive Auditory Cortex Recordings
The cerebral processing of voice information is known to engage, in human as well as non-human primates, “temporal voice areas” (TVAs) that respond preferentially to conspecific vocalizations. However, how voice information is represented by neuronal populations in these areas, particularly speaker identity information, remains poorly understood. Here, we used a deep neural network (DNN) to generate a high-level, small-dimension representational space for voice identity—the ‘voice latent space’ (VLS)—and examined its linear relation with cerebral activity via encoding, representational similarity, and decoding analyses. We find that the VLS maps onto fMRI measures of cerebral activity in response to tens of thousands of voice stimuli from hundreds of different speaker identities and better accounts for the representational geometry for speaker identity in the TVAs than in A1. Moreover, the VLS allowed TVA-based reconstructions of voice stimuli that preserved essential aspects of speaker identity as assessed by both machine classifiers and human listeners. These results indicate that the DNN-derived VLS provides high-level representations of voice identity information in the TVAs.
- Open Access
- 9 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 228-237
Towards the generation of synchronized and believable non-verbal facial behaviors of a talking virtual agent
This paper introduces a new model to generate rhythmically relevant non-verbal facial behaviors for virtual agents while they speak. The model demonstrates perceived performance comparable to behaviors directly extracted from the data and replayed on a virtual agent, in terms of synchronization with speech and believability. Interestingly, we found that training the model with two different sets of data, instead of one, did not necessarily improve its performance. The expressiveness of the people in the dataset and the shooting conditions are key elements. We also show that employing an adversarial model, in which fabricated fake examples are introduced during the training phase, increases the perception of synchronization with speech. A collection of videos demonstrating the results and code can be accessed at: https://github.com/aldelb/non_verbal_facial_animation.
- 9 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 228-237
Open AccessTowards the generation of synchronized and believable non-verbal facial behaviors of a talking virtual agent
This paper introduces a new model to generate rhythmically relevant non-verbal facial behaviors for virtual agents while they speak. The model demonstrates perceived performance comparable to behaviors directly extracted from the data and replayed on a virtual agent, in terms of synchronization with speech and believability. Interestingly, we found that training the model with two different sets of data, instead of one, did not necessarily improve its performance. The expressiveness of the people in the dataset and the shooting conditions are key elements. We also show that employing an adversarial model, in which fabricated fake examples are introduced during the training phase, increases the perception of synchronization with speech. A collection of videos demonstrating the results and code can be accessed at: https://github.com/aldelb/non_verbal_facial_animation.
- Open Access
- 1 July 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Transfer Learning by Weighting Convolution
Transferring pretrained deep architectures to datasets with few labels is still a challenge in many real-world situations. This paper presents a new framework to understand convolutional neural networks, by establishing connections between Kronecker factorization and convolutional layers. We then introduce Convolution Weighting Layers that learn a vector of weights for each channel, allowing efficient transfer learning in small training settings, as well as enabling pruning the transferred models. Experiments are conducted on two main settings with few labeled data: transfer learning for classification and transfer learning for retrieval. Two well known convolutional architectures are evaluated on five public datasets. We show that weighting convolutions is efficient to adapt pretrained models to new tasks and that pruned networks conserve good performance.
- 1 July 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Open AccessTransfer Learning by Weighting Convolution
Transferring pretrained deep architectures to datasets with few labels is still a challenge in many real-world situations. This paper presents a new framework to understand convolutional neural networks, by establishing connections between Kronecker factorization and convolutional layers. We then introduce Convolution Weighting Layers that learn a vector of weights for each channel, allowing efficient transfer learning in small training settings, as well as enabling pruning the transferred models. Experiments are conducted on two main settings with few labeled data: transfer learning for classification and transfer learning for retrieval. Two well known convolutional architectures are evaluated on five public datasets. We show that weighting convolutions is efficient to adapt pretrained models to new tasks and that pruned networks conserve good performance.
- 1 June 2010
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 19493983,p. 1-5
The LIGVID System for video retrieval and concept annotation
The LIGVID system is designed for online interactive video shots retrieval and annotation. It uses a user-controlled combination of multiple criteria: keywords, phonetic string, similarity to example images, semantic categories, and relevance feedback strategies: visual and temporal similarity to already identified positive images. In addition to Relevance Feedback processes, the system runs in background an active learning algorithm to better model the user's information need. Previous participation to video retrieval challenges has permit to show the effectiveness of our system.
- 1 June 2010
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 19493983,p. 1-5
The LIGVID System for video retrieval and concept annotation
The LIGVID system is designed for online interactive video shots retrieval and annotation. It uses a user-controlled combination of multiple criteria: keywords, phonetic string, similarity to example images, semantic categories, and relevance feedback strategies: visual and temporal similarity to already identified positive images. In addition to Relevance Feedback processes, the system runs in background an active learning algorithm to better model the user's information need. Previous participation to video retrieval challenges has permit to show the effectiveness of our system.
- 1 November 2009
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 15224880,p. 277-280
Efficient image concept indexing by harmonic & arithmetic profiles entropy
We propose new efficient visual features called Profile Entropy Features (PEF), giving information on the structure of the image content, and defined as the entropy of the distribution of a projection of the pixels. We analyse two simple projection operators (arithmetic or harmonic mean), and two orientations (horizontal and vertical). PEF are fast to compute (10 images per sec. on a PentiumIV) and of small dimension. Moreover, we show on High Level Feature task in TrecVid2008 that PEF performs in average better than the features of the state of the art (usual color features, edge direction, Gabor, and Local Binary Pattern). Moreover, we show on another international image retrieval campaign, the Visual Concept Detection of ImageCLEF2008, that the arithmetic and harmonic projections give complementary informations, yielding to the third best rank system in the official run of this campaign. Other properties of the PEF are discussed.
- 1 November 2009
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 15224880,p. 277-280
Efficient image concept indexing by harmonic & arithmetic profiles entropy
We propose new efficient visual features called Profile Entropy Features (PEF), giving information on the structure of the image content, and defined as the entropy of the distribution of a projection of the pixels. We analyse two simple projection operators (arithmetic or harmonic mean), and two orientations (horizontal and vertical). PEF are fast to compute (10 images per sec. on a PentiumIV) and of small dimension. Moreover, we show on High Level Feature task in TrecVid2008 that PEF performs in average better than the features of the state of the art (usual color features, edge direction, Gabor, and Local Binary Pattern). Moreover, we show on another international image retrieval campaign, the Visual Concept Detection of ImageCLEF2008, that the arithmetic and harmonic projections give complementary informations, yielding to the third best rank system in the official run of this campaign. Other properties of the PEF are discussed.