A Multimodal Fusion Depression Recognition Assisted Decision-Making System Based on EEG and Speech Signals
Authors:
- 18 October 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Abstract
Depression, as a widely prevalent mental disorder, significantly impacts the psychological well-being of a substantial portion of the global population. However, existing diagnostic methods for depression possess certain limitations, rendering them potentially ineffective for large-scale depression detection or failing to adequately exploit the multimodal physiological and behavioral information of individuals with depressive disorders. To address these challenges, this study proposes an innovative approach involving the multimodal fusion of Electroencephalography (EEG) and speech signals, aiming to comprehensively leverage data for the decision-making process. From aspects such as time-frequency domain features, spatial features, and semantic features, this research extracts information and fuses it to construct a multi-modal feature space. Simultaneously, a feature-level fusion strategy is introduced and applied in practical scenarios, resulting in the creation of a multi-modal fusion depression recognition assisted decision-making system. This system assists clinical practitioners in making more comprehensive and reliable identifications and diagnoses of individuals with depressive disorders. Following rigorous testing of its functionality, the multi-modal recognition system exhibits feasibility and effectiveness in real-world medical environments, offering novel perspectives and efficacious methodologies for the detection of depressive disorders.Keywords
Funding Information
- Natural Science Foundation of China (61971084,62025105,62001073,62221005,62272075,cstc2021ycjhbgzxm0039,CSTB2022BSXM-JCX0109,CSTB2022BSXMJCX0110,cx2021003,cx2021053)
This publication has 22 references indexed in Scilit:
- An Effective and Robust Framework for Ocular Artifact Removal From Single-Channel EEG Signal Based on Variational Mode DecompositionIEEE Sensors Journal, 2019
- Mobile Edge Computing-Enabled Internet of Vehicles: Toward Energy-Efficient SchedulingIEEE Network, 2019
- Efficient Epileptic Seizure Prediction Based on Deep LearningIEEE Transactions on Biomedical Circuits and Systems, 2019
- Vehicular Fog Computing: Enabling Real-Time Traffic Management for Smart CitiesIEEE Wireless Communications, 2019
- Integrating Deep and Shallow Models for Multi-Modal Depression Analysis—Hybrid ArchitecturesIEEE Transactions on Affective Computing, 2018
- Offloading in Internet of Vehicles: A Fog-Enabled Real-Time Traffic Management SystemIEEE Transactions on Industrial Informatics, 2018
- Separation of Sources From Single-Channel EEG Signals Using Independent Component AnalysisIEEE Transactions on Instrumentation and Measurement, 2017
- Prediction of fatigue-related driver performance from EEG data by deep Riemannian model2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2017
- Vehicular Social Networks: Enabling Smart MobilityIEEE Communications Magazine, 2017
- Presenting efficient features for automatic CAP detection in sleep EEG signals2015