Claudia Roswandowitz

The formation of social first impressions from voices is a central component of everyday social interactions. While past research has primarily investigated the effect of bottom‐up voice acoustic on social voice judgements, here we widen the perspective and investigate how bottom‐up acoustic and top‐down interpersonal similarity interactively influence social voice judgements. We quantified acoustic distance to an average voice as a measure of acoustic typicality and combined this with interpersonal similarity between each listener‐speaker pair derived from Big Five personality traits. 110 listeners rated 30 words spoken by female speakers on attractiveness, likability, trustworthiness, and dominance, and data were analysed using Generalised Additive Mixed Models to capture nonlinear and interactive effects. Across social dimensions, acoustic typicality emerged as the strongest predictor of social judgements, with more typical voices receiving the most positive ratings. However, social judgements were not determined by acoustic cues alone. For attractiveness, likability and trustworthiness, similar personalities between speaker and listener enhanced social evaluations, particularly for acoustically atypical voices. In contrast, dominance judgements were exclusively explained by acoustic typicality without significant modulation by interpersonal similarity. Together, these results show that most social voice judgements arises from a dyadic, context‐sensitive integration of acoustic information and interpersonal similarity.

Human listeners are remarkably skilled at identifying voices, yet the mental representations that support voice identity processing remain poorly understood. In this study, we present a data-driven approach that reveals core dimensions of the voice identity space. We compiled a large-scale voice corpus including 337 male speakers producing each a unique sentence (VocDat). Using a Sparse Positive Similarity Embedding (SPoSE) model trained on voice similarity judgments from 389 participants, we identified a five-dimensional voice identity space capturing human between-speaker similarity judgments. Regression analyses showed that these model dimensions were characterised by distinct combinations of acoustic voice properties, with pitch, spectral slope, and syllable rate emerging as the most consistent and informative features. These acoustic labels closely aligned with perceptual voice attributes provided by trained phoneticians (e.g. low pitch, old age, fast speech), demonstrating that these dimensions can be interpreted in a perceptually meaningful way. By combining a data-driven modelling approach with a large and acoustically diverse speaker sample, and by characterising voice identity space dimensions in both acoustic and perceptual terms, our findings provide perceptually grounded insights into the structure of voice identity representations.

Deepfakes are viral ingredients of digital environments, and they can trick human cognition into misperceiving the fake as real. Here, we test the neurocognitive sensitivity of 25 participants to accept or reject person identities as recreated in audio deepfakes. We generate high-quality voice identity clones from natural speakers by using advanced deepfake technologies. During an identity matching task, participants show intermediate performance with deepfake voices, indicating levels of deception and resistance to deepfake identity spoofing. On the brain level, univariate and multivariate analyses consistently reveal a central cortico-striatal network that decoded the vocal acoustic pattern and deepfake-level (auditory cortex), as well as natural speaker identities (nucleus accumbens), which are valued for their social relevance. This network is embedded in a broader neural identity and object recognition network. Humans can thus be partly tricked by deepfakes, but the neurocognitive mechanisms identified during deepfake processing open windows for strengthening human resilience to fake information.

Voice signals are relevant for auditory communication and suggested to be processed in dedicated auditory cortex (AC) regions. While recent reports highlighted an additional role of the inferior frontal cortex (IFC), a detailed description of the integrated functioning of the AC–IFC network and its task relevance for voice processing is missing. Using neuroimaging, we tested sound categorization while human participants either focused on the higher‐order vocal‐sound dimension (voice task) or feature‐based intensity dimension (loudness task) while listening to the same sound material. We found differential involvements of the AC and IFC depending on the task performed and whether the voice dimension was of task relevance or not. First, when comparing neural vocal‐sound processing of our task‐based with previously reported passive listening designs we observed highly similar cortical activations in the AC and IFC. Second, during task‐based vocal‐sound processing we observed voice‐sensitive responses in the AC and IFC whereas intensity processing was restricted to distinct AC regions. Third, the IFC flexibly adapted to the vocal‐sounds' task relevance, being only active when the voice dimension was task relevant. Forth and finally, connectivity modeling revealed that vocal signals independent of their task relevance provided significant input to bilateral AC. However, only when attention was on the voice dimension, we found significant modulations of auditory‐frontal connections. Our findings suggest an integrated auditory‐frontal network to be essential for behaviorally relevant vocal‐sounds processing. The IFC seems to be an important hub of the extended voice network when representing higher‐order vocal objects and guiding goal‐directed behavior.

The voice contains elementary social communication cues, conveying speech, as well as paralinguistic information pertaining to the emotional state and the identity of the speaker. In contrast to vocal-speech and vocal-emotion processing, voice-identity processing has been less explored. This seems surprising, given the day-to-day significance of person recognition by voice. A valuable approach to unravel how voice-identity processing is accomplished is to investigate people who have a selective deficit in recognising voices. Such a deficit has been termed phonagnosia. In the present chapter, we provide a systematic overview of studies on phonagnosia and how they relate to current neurocognitive models of person recognition. We review studies that have characterised people who suffer from phonagnosia following brain damage (i.e. acquired phonagnosia) and also studies, which have examined phonagnosia cases without apparent brain lesion (i.e. developmental phonagnosia). Based on the reviewed literature, we emphasise the need for a careful behavioural characterisation of phonagnosia cases by taking into consideration the multistage nature of voice-identity processing and the resulting behavioural phonagnosia subtypes.

Recognizing the identity of others by their voice is an important skill for social interactions. To date, it remains controversial which parts of the brain are critical structures for this skill. Based on neuroimaging findings, standard models of person-identity recognition suggest that the right temporal lobe is the hub for voice-identity recognition. Neuropsychological case studies, however, reported selective deficits of voice-identity recognition in patients predominantly with right inferior parietal lobe lesions. Here, our aim was to work towards resolving the discrepancy between neuroimaging studies and neuropsychological case studies to find out which brain structures are critical for voice-identity recognition in humans. We performed a voxel-based lesion-behaviour mapping study in a cohort of patients (n = 58) with unilateral focal brain lesions. The study included a comprehensive behavioural test battery on voice-identity recognition of newly learned (voice-name, voice-face association learning) and familiar voices (famous voice recognition) as well as visual (face-identity recognition) and acoustic control tests (vocal-pitch and vocal-timbre discrimination). The study also comprised clinically established tests (neuropsychological assessment, audiometry) and high-resolution structural brain images. The three key findings were: (i) a strong association between voice-identity recognition performance and right posterior/mid temporal and right inferior parietal lobe lesions; (ii) a selective association between right posterior/mid temporal lobe lesions and voice-identity recognition performance when face-identity recognition performance was factored out; and (iii) an association of right inferior parietal lobe lesions with tasks requiring the association between voices and faces but not voices and names. The results imply that the right posterior/mid temporal lobe is an obligatory structure for voice-identity recognition, while the inferior parietal lobe is only a facultative component of voice-identity recognition in situations where additional face-identity processing is required.

People with autism spectrum disorder (ASD) have difficulties in identifying another person by face and voice. This might contribute considerably to the development of social cognition and interaction difficulties. The characteristics of the voice recognition deficit in ASD are unknown. Here, we used a comprehensive behavioral test battery to systematically investigate voice processing in high‐functioning ASD (n = 16) and typically developed pair‐wise matched controls (n = 16). The ASD group had particular difficulties with discriminating, learning, and recognizing unfamiliar voices, while recognizing famous voices was relatively intact. Tests on acoustic processing abilities showed that the ASD group had a specific deficit in vocal pitch perception that was dissociable from otherwise intact acoustic processing (i.e., musical pitch, musical, and vocal timbre perception). Our results allow a characterization of the voice recognition deficit in ASD: The findings indicate that in high‐functioning ASD, the difficulty to recognize voices is particularly pronounced for learning novel voices and the recognition of unfamiliar peoples’ voices. This pattern might be indicative of difficulties with integrating the acoustic characteristics of the voice into a coherent percept—a function that has been previously associated with voice‐selective regions in the posterior superior temporal sulcus/gyrus of the human brain. Autism Res 2017, 10: 155–168. © 2016 International Society for Autism Research, Wiley Periodicals, Inc.