Jennifer Hamet Bagnou

Social performance in interaction is typically assessed through global observational ratings, yet the behavioural bases underlying these judgements remain poorly specified. This paper examines multimodal behavioural indicators of perceived social performance in dyadic collaborative problem-solving (CPS), using the TALENT video corpus of dyadic interaction. Linear regression models related verbal and facial indicators to ratings on two standardised scales: the Social Performance Rating Scale (SPRS) and the Social Skills of Collaboration scale (SSC). Results show that verbal engagement (e.g., longer turn duration) and collective coordination behaviours (group references, coordination initiatives, task-relevant contributions) are positively associated with higher social performance ratings on both scales. Facial Action Unit effects are context-dependent: the direction and magnitude of AU associations vary as a function of the co-occurring verbal behaviour, with no single facial marker consistently predicting performance. These findings provide an empirically grounded, multimodal characterisation of perceived social performance in CPS, with implications for the computational modelling of social signals in collaborative interactive systems.

Les aptitudes sociales, comportementssocialement appris, permettent aux individus des’adapter dans diverses situations sociales. Parmielles, les aptitudes de Résolution Collaborative deProblèmes (RCP) occupent une place importantepour la réussite académique et professionnelle.Elles sont définies comme un ensembled’aptitudes à résoudre des problèmes et àprogresser vers un objectif commun tout encollaborant avec les membres du groupe. Nousappréhendons leur mobilisation à travers de laperformance sociale, indicateur de l’adéquationdes réponses d’un individu aux attentes de lasituation. Cette performance résulte d’unedynamique entre facteurs endogènes (liés à lapersonne) et exogènes (liés à l’environnement).Cette thèse examine comment la performance enRCP est modulée par, d’une part, l’anxiété socialeet l’alexithymie, et, d’autre part, par une inductiond’attention focalisée sur soi ainsi que par lecomportement d’un partenaire virtuel. Pour cefaire, nous avons constitué un corpus multimodalde dyades engagées dans des tâchescollaboratives avec un feedback visuel manipulé(retour sur soi, sur l’autre, ou combiné). À partir dece corpus, nous avons validé deux échellesfrancophones permettant d’évaluer les aptitudessociales spécifiques à la RCP.Nos résultats montrent que l’attention centréesur soi peut nuire aux performances sociales,surtout chez les personnes alexithymiques ;toutefois, chez les participants très anxieux, lesperformances étaient préservées, voireaméliorées, lorsque l’attention était centrée sursoi plutôt que répartie entre soi et le partenaire.Pour isoler l’effet du partenaire dans des tâchesde RCP, nous avons conçu des agents virtuels(profil performant vs non-performant). Interagiravec un agent performant accroît laperformance sociale de l’utilisateur, alors qu’unagent non-performant la diminue. Les résultatsde cette seconde étude suggèrent que ceseffets sont modérés par l’anxiété etl’alexithymie. Les contributions de cette thèseapportent de nouvelles connaissances surl’articulation entre facteurs endogènes etexogènes de la performance sociale en RCP,apportent des outils de mesure robustes (SPRS,SSC), un corpus partageable et ouvrent desperspectives applicatives via des agents virtuelsadaptatifs pour l’entraînement personnalisé desaptitudes sociales.

Background: Social communication is a crucial factor influencing human social life. Quantifying the degree of difficulty faced in social communication is necessary for understanding developmental and neurological disorders and for creating systems used in automatic symptom screening and assistive methods such as social skills training (SST). SST by a human trainer is a well-established method. Previous SST used a modified roleplay test to evaluate human social communication skills. However, there are no widely accepted evaluation criteria or social behavioral markers to quantify social performance during SST. Objective: This paper has 2 objectives. First, we propose applying the Social Performance Rating Scale (SPRS) to SST data to measure social communication skills. We constructed a Japanese version of the SPRS already developed in English and French. Second, we attempt to quantify action units during SST for people with autism spectrum disorder (ASD) or schizophrenia. Methods: We used videos of interactions between trainers, adults with ASD (n=16) or schizophrenia (n=15), and control participants (n=19) during SST sessions. Two raters applied the proposed scale to annotate the collected data. We investigated the differences between roleplay tasks and participant groups (ASD, schizophrenia, and control). Furthermore, the intensity of action units on the OpenFace toolkit was measured in terms of mean and SD during SST roleplaying. Results: We found significantly greater gaze scores in adults with ASD than in adults with schizophrenia. Differences were also found between the ratings of different tasks in the adults with schizophrenia and the control participants. Action units numbered AU06 and AU12 were significantly deactivated in people with schizophrenia compared with the control group. Moreover, AU02 was significantly activated in people with ASD compared with the other groups. Conclusions: The results suggest that the SPRS can be a useful tool for assessing social communication skills in different cultures and different pathologies when used with the modified roleplay test. Furthermore, facial expressions could provide effective social and behavioral markers to characterize psychometric properties. Possible future directions include using the SPRS for assessing social behavior during interaction with a digital agent.

Huntington’s Disease (HD) is an inherited neurodegenerative disease characterized by a combination of motor, cognitive, and behavioral disorders. The social and behavioral symptoms observed in HD patients impact their quality of life and probably explain their relational difficulties, conflicts, and social withdrawal. In this study, we described the development of the Social Relationship Self-Questionnaire (SRSQ), a self-reporting questionnaire that assesses how HD patients perceived their social relationships. The scale was proposed for 66 HD patients at an early stage of the disease, 32 PreHD patients (individuals carrying the mutant gene without motor symptoms), and 66 controls. The HD patients were included in a prospective longitudinal follow-up for an average of 1.07 years with motor, functional, cognitive, and behavioral assessments. Based on the HD patients’ answers at baseline, we identified two domains in the SRSQ. The first domain was related to social motivation and correlated with cognitive performance. The second domain was related to emotional insight and correlated with behavioral symptoms such as apathy, anxiety, and irritability. We discovered that both SRSQ domain scores at baseline predicted future motor, functional, and cognitive decline in HD.