JYJia Yao

Jia Yao

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Jia Yao has a total of 5 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • MMFC-PFL: Multimodal Feature Collaborative Personalized Federated Learning

    by Dongdong Li, Zhenqiang Weng, Jia Yao, Zhengji Xuan, Zhe Wang

    Federated learning (FL) aims to improve model performance while protecting user privacy. The amalgamation of multimodal data, encompassing diverse data types, introduces challenges in FL scenarios. To address these challenges, we propose an innovative multimodal feature collaborative personalized federated learning framework (MMFC-PFL) grounded in the unique characteristics of multimodal features. It includes multimodal collaborative loss and personalized parameter aggregation. The multimodal collaborative loss comprises intra-modal loss, aimed at proficiently representing unimodal features, and inter-modal loss, designed to seamlessly integrate collaborative global multimodal features. The process of personalized parameter aggregation entails the discerning selection of clients during initialization in each global round, with a specific emphasis on those that exhibit maximal separation from the current client for aggregation purposes. The distance between clients is facilitated through Canonical Correlation Analysis on inter-modal features. Additionally, we present an iterative client-wise personalized aggregation strategy that enhances the generalization capabilities of the personalized model. We validate the proposed MMFC-PFL on three datasets: AV-MNIST, MM-IMDB, and MIMIC III. The experimental results demonstrate the effective improvement of model performance and robustness by the proposed multimodal federated learning framework.


  • FAD3QN: A Brain-Inspired Deep Reinforcement Learning Model for Speech Depression Detection

    by Dongdong Li, Jia Yao, Zhe Wang, Yichao Yi

    In recent years, the high prevalence and severity of depression have highlighted the urgent need for early detection. Depressed patients exhibit noticeable emotional changes in their speech, but existing detection methods face significant challenges in modeling emotion perception mechanisms. In this article, inspired by the knowledge of reinforcement learning neuroscience and the theory of emotion perception in the limbic system, we propose a brain-inspired model frontal-amygdala double dueling deep Q network for depression detection based on speech. The model simulates the frontal lobe and amygdala-centred brain mechanisms for emotion perception through the reinforcement learning framework of double dueling deep Q-networks, and embeds the skip-connected 1-D convolutional neural network and bidirectional long short-term memory network neural networks to simulate the emotion perception process in the limbic system. In addition, we designed an adaptively tuned reward function to address the data imbalance in depression detection, and incorporated an additional step-size penalty factor to limit the number of incorrect decisions made by the agent during the early stages of training. Experimental results across multiple datasets demonstrate the effectiveness and generalizability of our approach. Meanwhile, the relevant ablation experiments conducted in this article validate the key roles of the frontal and limbic systems in the reward learning and emotion perception process, as well as the importance of the adaptive reward function in solving the data imbalance problem.