Valentin Durand de Gevigney

Modèles d'apprentissage automatique pour la détection multimodale de comportements anormaux: Nos travaux portent sur le développement de modèles pour la détection de comportements humains anormaux en utilisant plusieurs modalités (e.g. audio, vidéo, etc.) La détection de comportements anormaux a de nombreuses applications, de la sécurité dans les lieux publics au vote électronique en passant par la santé mentale et les handicaps invisibles. Nous commençons d’abord par définir le domaine dans lequel s’ancre cette thèse en prenant soin d’en tracer les contours. Puisque nous traitons du comportement humain, nous donnons quelques mises en gardes sur le fait de considérer un comportement comme anormal et les potentielles applications. Ensuite, nous faisons un tour d’horizon de la littérature qui se rapporte à notre sujet. Nous en identifions des limites que nous avons tenté de repousser pendant cette thèse. Nos résultats se divisent en trois catégories : 1) Trois nouvelles méthodes pour détection de comportements anormaux. La première méthode modélise les comportements comme des mécanismes temporels, la seconde y ajoute un cadre probabiliste et la troisième se base sur la théorie de l’information et l’entropie de Shannon. 2) Des méthodes de modélisation multimodales afin de pallier au manque dans l’état de l’art. 3) Des méthodes pour la modélisation de représentations du comportement humain. Nous terminons avec une vue d’ensemble de nos travaux et de nos trouvailles, observant les nouvelles limites du domaine et définissant les perspectives pour de nouveau repousser ces limites.

Dopamine transporter (DAT) SPECT is a validated biomarker for Parkinson’s disease (PD) and related degenerative parkinsonisms. Interpretation relies on visual assessment supported by striatal image features such as striatal binding ratios (SBRs). Deep learning can aid this process but often underuses complementary data, lacks robustness to heterogeneous inputs, and offers limited interpretability. We built an end-to-end multimodal framework that encodes DAT images and scalar data (patient age and striatal image features) using, respectively, a vision transformer and a multilayer perceptron. A transformer-based fusion module then combined the encoded representations, while tackling possible missing inputs. Interpretability was provided through modality-level attention, spatial attention maps, occlusion analysis, and scalar feature saliency. Performance was evaluated on 664 Parkinson’s Progression Markers Initiative (PPMI) cases and two local datasets A ( N = 228) and B ( N = 530) from different devices, including PD, atypical parkinsonisms, and non-degenerative subjects. Transfer learning involved pretraining on two datasets and finetuning on the third. On PPMI, the model reached 97.4% AUROC, 95.5% accuracy, 97.0% sensitivity, and 91.9% specificity, matching state-of-the-art performance. Results were similar on dataset B (98.6% AUROC) but lower on dataset A (92.6% AUROC), likely due to its smaller size and reduced image quality. Explainability analyses showed the model focused on clinically relevant striatal regions and identified key scalar features such as putamen SBR and asymmetry. The fusion module also supported stable predictions despite missing data. Our method efficiently combined multimodal data with heterogeneous datasets and partial multimodal data. Integrated explainability tools showed clinically meaningful attention that is expected to favor its adoption.

Relating explicit psychological mechanisms and observable behaviours is a central aim of psychological and behavioural science. One of the challenges is to understand and model the role of consciousness and, in particular, its subjective perspective as an internal level of representation (including for social cognition) in the governance of behaviour. Toward this aim, we implemented the principles of the Projective Consciousness Model (PCM) into artificial agents embodied as virtual humans, extending a previous implementation of the model. Our goal was to offer a proof-of-concept, based purely on simulations, as a basis for a future methodological framework. Its overarching aim is to be able to assess hidden psychological parameters in human participants, based on a model relevant to consciousness research, in the context of experiments in virtual reality. As an illustration of the approach, we focused on simulating the role of Theory of Mind (ToM) in the choice of strategic behaviours of approach and avoidance to optimise the satisfaction of agents’ preferences. We designed a main experiment in a virtual environment that could be used with real humans, allowing us to classify behaviours as a function of order of ToM, up to the second order. We show that agents using the PCM demonstrated expected behaviours with consistent parameters of ToM in this experiment. We also show that the agents could be used to estimate correctly each other’s order of ToM. Furthermore, in a supplementary experiment, we demonstrated how the agents could simultaneously estimate order of ToM and preferences attributed to others to optimize behavioural outcomes. Future studies will empirically assess and fine tune the framework with real humans in virtual reality experiments.