YZYan Zhuang

Yan Zhuang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Yan Zhuang has a total of 27 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Automatic Social Popularity Prediction Agents for Multimodal Social Media Popularity Prediction

    by Weilong Chen, Kaige Zheng, Zhenyu Yang, Yan Zhuang, Bin Chen, Fuji Ren, Yanru Zhang


  • MC-FF: A multi-community feature fusion framework for social bot detection

    by Wei Li, Jiawen Deng, Jiali You, Yuanyuan He, Yan Zhuang, Fuji Ren

  • Intra-Sample and Intra-Modal Enhancement for Multimodal Sentiment Analysis With Missing Modalities

    by Yan Zhuang, Yanru Zhang, Jiawen Deng, Fuji Ren

    Multimodal sentiment analysis (MSA) with missing modalities involves understanding the person's sentiment using multimodal data where some modalities are missing. Most existing methods focus on reconstructing the missing modalities using the available modalities from each sample, relying on modality-common information. However, these methods overlook the modality-specific information that other samples can provide. Additionally, these approaches often require the guidance of full modality representations during the reconstruction process, which is impractical in resource-constrained real-world scenarios. To address these challenges, we propose the Intra-sample and Intra-modal Enhancement (IIE) framework. The IIE framework enhances both sample-level and modality-level representations to capture additional modality-common and modality-specific information from existing modalities, without requiring full modalities. Specifically, IIE first learns sample-level representations by distilling modality-common information from the available modalities into learnable latent units. Then, it enhances modality-level representations by leveraging modality-specific information from other samples with the same modality, which is crucial for improving robustness in the presence of missing modalities. Finally, IIE ensures consistency between the enhanced modality-level and sample-level representations, combining the enhanced and initial representations to make predictions. Extensive experiments on three datasets demonstrate that the IIE framework significantly outperforms existing methods in terms of both effectiveness and robustness in handling MSA with missing modalities. Code is available at https://github.com/YetZzzzzz/IIE.


  • Open Access

    FAME: Fusion-Aware Multi-modal Ensemble for Social Media Popularity Prediction

    by Yan Zhuang, Wei Bai, Yanru Zhang, Minhao Liu, Jiawen Deng, Fuji Ren


  • Multi-Level Contrastive Learning for Multimodal Sentiment Analysis

    by Yan Zhuang, Wei Bai, Yanru Zhang, Jiawen Deng, Zheng Hu, Xiaoyue Zhang, Fuji Ren

    Multimodal sentiment analysis has garnered increasing attention. The bulk of existing work in multimodal sentiment analysis primarily focuses on designing various networks to align and subsequently fuse representations from individual modalities. Contrastive learning, recognized for its intrinsic alignment capabilities, has also been extensively applied in multimodal sentiment analysis. However, current contrastive learning methods are often limited to pairwise modalities and typically perform contrastive learning prior to modality fusion, neglecting the consistency of interactions across multiple modalities. Moreover, they overlook the overall consistency within samples. To address these issues, we introduce a novel Multi-Level Contrastive Learning (MLCL) framework for multimodal sentiment analysis, composed of Uni-Modal Contrastive Learning (UMCL), Bi-Modal Contrastive Learning (BMCL) and Tri-Modal Contrastive Learning (TMCL). UMCL enhances intra-modal representations by creating positive pairs using modality-specific random dropout, while BMCL leverages the asymmetry of attention mechanisms, using two directional attentions as positive samples. TMCL aligns non-overlapping uni-modal and bi-modal representations, underscoring the complementarity of tri-modal information. The effectiveness of MLCL is demonstrated through its performance on multiple datasets. Our comprehensive experiments across multiple datasets demonstrate the superiority of the MLCL framework, which achieves new state-of-the-art performance.


  • Enhanced Emotion Recognition in Conversations Through Hybrid Context Encoding and Latent Dependency Mining

    by Zheng Hu, Jiawen Deng, Satoshi Nakagawa, Yan Zhuang, Xiaoyue Zhang, Shimin Cai, Fuji Ren

    Emotion recognition in conversations (ERC) is a pivotal component of affective computing, involving a common two-stage paradigm where pre-trained language models first extract context-independent features, followed by the encoding of contextual information and the modeling of emotional dependencies. This paradigm faces two challenges: (1) Existing methods struggle to capture both the intra-dialogue emotional continuity and the inter-dialogue semantic similarity. (2) The complexity of emotional elicitation processes gives rise to entangled dependencies, termed “latent dependencies”, which are difficult for current methods to detect and analyze. To overcome these challenges, we propose a Hybrid-Context Encoder with an Automated Latent Dependency Mining model for ERC. Specifically, we examine the emotional continuity and the semantic similarity from the standpoint of context encoders. We experimentally find that context encoders with different architectures exhibit distinct benefits. Based on these findings, we design a hybrid contextual encoding module that effectively combines the strengths of various encoders. Additionally, we design a lightweight generative module for latent dependency mining that autonomously generates a context mask, enabling the effective discovery of latent dependencies. We conduct extensive experiments on three datasets in the text modality. Our model achieves the best performance, which validates the superiority of our approach.


  • Open Access

    R3DG: Retrieve, Rank, and Reconstruction with Different Granularities for Multimodal Sentiment Analysis

    by Yan Zhuang, Yanru Zhang, Jiawen Deng, Fuji Ren

    Multimodal sentiment analysis (MSA) aims to assess emotional states by integrating information from text, audio, and video. However, the heterogeneous nature of these modalities presents substantial challenges for accurate sentiment prediction. Existing approaches typically align pairs of modalities using attention mechanisms or contrastive learning, which are computationally expensive. Additionally, they often rely on a single granularity of alignment, either by averaging features over all time steps or aligning features at each individual time step. These approaches overlook the fact that emotional expression can vary across individuals and contexts, requiring multiple granularities to capture emotion effectively. To address these challenges, we propose a novel framework, Retrieve, Rank, and Reconstruction with Different Granularities (R3DG). R3DG segments the audio and video modalities into multiple representations at varying granularities based on their temporal durations. It then selects the most relevant representations that align closely with the text modality. To preserve the original information, R3DG reconstructs the audio and video data using the selected representations. Finally, the fused audio, video, and text features are aligned and combined for sentiment prediction, reducing the need for multiple alignment steps. Extensive experiments on 5 benchmark MSA datasets demonstrate that R3DG outperforms existing methods and achieves substantial reductions in computational time. Code is available at https://github.com/YetZzzzzz/R3DG .


  • Hierarchical Denoising for Robust Social Recommendation

    by Zheng Hu, Satoshi Nakagawa, Yan Zhuang, Jiawen Deng, Shimin Cai, Tao Zhou, Fuji Ren

    Social recommendations leverage social networks to augment the performance of recommender systems. However, the critical task of denoising social information has not been thoroughly investigated in prior research. In this study, we introduce a hierarchical denoising robust social recommendation model to tackle noise at two levels: 1) intra-domain noise, resulting from user multi-faceted social trust relationships, and 2) inter-domain noise, stemming from the entanglement of the latent factors over heterogeneous relations (e.g., user-item interactions, user-user trust relationships). Specifically, our model advances a preference and social psychology-aware methodology for the fine-grained and multi-perspective estimation of tie strength within social networks. This serves as a precursor to an edge weight-guided edge pruning strategy that refines the model's diversity and robustness by dynamically filtering social ties. Additionally, we propose a user interest-aware cross-domain denoising gate, which not only filters noise during the knowledge transfer process but also captures the high-dimensional, nonlinear information prevalent in social domains. We conduct extensive experiments on three real-world datasets to validate the effectiveness of our proposed model against state-of-the-art baselines. We perform empirical studies on synthetic datasets to validate the strong robustness of our proposed model.