Lili Guo

Multimodal Emotion Recognition in Conversation (MERC) aims to recognize emotions in conversational videos by fusing multimodal information. Current methods mainly rely on the direct fusion of emotional representations. This simple concatenation or linear combination not only fails to fully explore the deep semantic correlations and cross-modal dependencies among visual, textual, audio and other modalities, but also causes redundant feature interference due to the heterogeneity and misalignment of features from different modalities. Therefore, this paper proposes the Graph Dynamic Fusion Network with Modality Mutual Information Maximization (MMGDFN), designed to effectively exploit the complementarity among modalities in conversations. Specifically, by constructing dynamic graph networks to model the interaction relationships among features of various modalities, the method adaptively captures cross-modal dependencies through graph structures. Meanwhile, a mutual information maximization mechanism is introduced to quantify and enhance complementary information among modalities, enabling adaptive balancing of multimodal complementary information and effective filtering of redundant features. Experimental results on two publicly available datasets (i.e. IEMOCAP and MELD) demonstrate that the proposed MMGDFN significantly outperforms state-of-the-art multimodal emotion recognition methods.

Contrastive learning has strong generalization ability and the capability to learn automatically without labeled information. However, it still faces challenges such as insufficient feature diversity, a lack of multi-level semantics, and the balance between tolerance and consistency. To address these challenges, This study propose a Multi-view Hierarchical Graph Contrastive Learning method. First, a new view is generated through a diffusion matrix to provide multi-view data for contrastive learning. Then, these multi-view data are fed into an asynchronous asymmetric network structure, specifically using graph network models to learn diversified features. Next, we adopt a self-designed hierarchical contrastive learning framework, constructing a three-level contrastive loss for joint optimization of nodes, subgraphs, and global graphs. Meanwhile, we introduce alignment and consistency and appropriately adjust the loss function through a temperature coefficient. Ultimately, the model achieves excellent classification performance on multiple datasets through node classification and graph classification tasks.

As a new paradigm that integrates clustering with federated learning, federated clustering (FC) has recently attracted increasing attention, as it addresses the practical issue of privacy protection in distributed data. In this paper, we provide a comprehensive survey of recent advances in FC. This survey is organized into four parts. First, since FC is often developed by extending existing clustering methods, we review several classical clustering paradigms. Meanwhile, the inherent challenges of FC are summarized, and common improvement strategies are categorized. Second, we summarize experimental setups and evaluation protocols used in FC studies. Third, from the perspectives of data partitioning schemes and whether deep representation learning is incorporated, FC methods are divided into four categories, and representative algorithms in each category are reviewed. Finally, we discuss the limitations of current FC approaches and highlight potential directions for future research.

Emotion Recognition in Conversation (ERC) enables machines to respond emotionally to users' requests, which is crucial in human-computer interaction. Existing methods concentrate on capturing context and speaker-dependent relationships within the text modality or emphasize multi-modal interaction and feature fusion. However, these methods fail to effectively align shared emotional representations across modalities and preserve unique emotional features within each modality. To address this challenge, we propose an end-to-end framework called Cross-Modal Shared Attention (ShareERC). Specifically, we first extract context-aware features from text and audio, aggregating emotional information within each modality. Leveraging this aggregated emotional information, we design a Cross-Modal Shared Attention module that aligns shared features across modalities in the feature space while preserving the distinct features of each modality. Extensive experiments on the MELD and IEMOCAP datasets demonstrate the effectiveness and superiority of the proposed ShareERC.

Image clustering is a crucial but open and challenging task in machine learning and computer vision. Deep image clustering methods have made significant advancements in largescale and high-dimensional image datasets, but they only explore clusters according to feature similarity. How to improve the semantic plausibility of these clusters remains a challenging problem. To address this problem, we propose a joint semantic image clustering (SIC) with deep learning framework. Our key idea is to explore semantic clusters from both instance-level and clusterlevel perspectives. At the cluster level, we assume that the highest-confidence partitions possess reasonable semantic divisions; at the instance level, we assume that neighboring samples belong to the same semantic category. With the constraints of the cluster level and instance level, the intracluster compactness and interclass discrepancy are increased. Clustering results heavily rely on the quality of image representations. Therefore, we introduce strong data augmentations with three shared-weight backbone networks to learn the most inherent features. The experimental results on various image datasets demonstrate our framework’s superiority over a wide range of state-of-the-art approaches.

Emotion Recognition in Conversation (ERC) aims to detect the emotions of individual utterances within a conversation. Generating efficient and modality-specific representations for each utterance remains a significant challenge. Previous studies have proposed various models to integrate features extracted using different modality-specific encoders. However, they neglect the varying contributions of modalities to this task and introduce high complexity by aligning modalities at the frame level. To address these challenges, we propose the Multi-modal Anchor Gated Transformer with Knowledge Distillation (MAGTKD) for the ERC task. Specifically, prompt learning is employed to enhance textual modality representations, while knowledge distillation is utilized to strengthen representations of weaker modalities. Furthermore, we introduce a multi-modal anchor gated transformer to effectively integrate utterance-level representations across modalities. Extensive experiments on the IEMOCAP and MELD datasets demonstrate the effectiveness of knowledge distillation in enhancing modality representations and achieve state-of-the-art performance in emotion recognition. Our code is available at: https://github.com/JieLi-dd/MAGTKD.

In image denoising research, discriminative models have achieved impressive results which mainly owes to the powerful ability of convolutional networks in local feature extraction. However, there is still room for improvement due to insufficient utilization of global information. Although using fully connected layers or increasing network depth can supplement global information, this results in a significant increase in parameters and computational cost. To address these issues, we propose a global information compensation network (GICN) for image denoising in this paper. Firstly, at the shallow network part, we propose a global feature mining block that enhances the network's ability to extract global information by combining non-local blocks and the Fourier transform while improving the interpretability of the model. Secondly, between the encoder and decoder, we propose a cross-scale feature aggregation block to fuse information at different scales. Finally, we employ attention blocks to improve skip connections to better capture long-distance dependencies. Extensive experimental results show that our proposed GICN effectively compensates for global information, achieves a balance between denoising efficiency and effect, and surpasses mainstream methods in multiple benchmark tests.