Yan Zhuang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Yan Zhuang has a total of 27 co-authors. Here are the top co-authors by number of shared publications:
- Fuji Ren
University of Electronic Science and Technology of China, Chengdu, Sichuan, China
10 shared publications - Jiawen Deng
University of Electronic Science and Technology of China, Chengdu, China
6 shared publications - Yanru Zhang
University of Electronic Science and Technology of China
5 shared publications Jiawen Deng
3 shared publications- Zheng Hu
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China
3 shared publications - Jiali You
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China
2 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 17592-17598
Automatic Social Popularity Prediction Agents for Multimodal Social Media Popularity Prediction
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 17592-17598
Open AccessAutomatic Social Popularity Prediction Agents for Multimodal Social Media Popularity Prediction
- 22 August 2026
- Published by Elsevier in Computers & Security
- Vol. 171
MC-FF: A multi-community feature fusion framework for social bot detection
- 22 August 2026
- Published by Elsevier in Computers & Security
- Vol. 171
MC-FF: A multi-community feature fusion framework for social bot detection
- Open Access
- 15 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 1355-1364
ReNoRD: Learning from Relations under Noisy Pseudo Labels via Relational Distillation for Multimodal Sentiment
- 15 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 1355-1364
Open AccessReNoRD: Learning from Relations under Noisy Pseudo Labels via Relational Distillation for Multimodal Sentiment
- 18 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 1847-1859
Intra-Sample and Intra-Modal Enhancement for Multimodal Sentiment Analysis With Missing Modalities
Multimodal sentiment analysis (MSA) with missing modalities involves understanding the person's sentiment using multimodal data where some modalities are missing. Most existing methods focus on reconstructing the missing modalities using the available modalities from each sample, relying on modality-common information. However, these methods overlook the modality-specific information that other samples can provide. Additionally, these approaches often require the guidance of full modality representations during the reconstruction process, which is impractical in resource-constrained real-world scenarios. To address these challenges, we propose the Intra-sample and Intra-modal Enhancement (IIE) framework. The IIE framework enhances both sample-level and modality-level representations to capture additional modality-common and modality-specific information from existing modalities, without requiring full modalities. Specifically, IIE first learns sample-level representations by distilling modality-common information from the available modalities into learnable latent units. Then, it enhances modality-level representations by leveraging modality-specific information from other samples with the same modality, which is crucial for improving robustness in the presence of missing modalities. Finally, IIE ensures consistency between the enhanced modality-level and sample-level representations, combining the enhanced and initial representations to make predictions. Extensive experiments on three datasets demonstrate that the IIE framework significantly outperforms existing methods in terms of both effectiveness and robustness in handling MSA with missing modalities. Code is available at https://github.com/YetZzzzzz/IIE.
- 18 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 1847-1859
Intra-Sample and Intra-Modal Enhancement for Multimodal Sentiment Analysis With Missing Modalities
Multimodal sentiment analysis (MSA) with missing modalities involves understanding the person's sentiment using multimodal data where some modalities are missing. Most existing methods focus on reconstructing the missing modalities using the available modalities from each sample, relying on modality-common information. However, these methods overlook the modality-specific information that other samples can provide. Additionally, these approaches often require the guidance of full modality representations during the reconstruction process, which is impractical in resource-constrained real-world scenarios. To address these challenges, we propose the Intra-sample and Intra-modal Enhancement (IIE) framework. The IIE framework enhances both sample-level and modality-level representations to capture additional modality-common and modality-specific information from existing modalities, without requiring full modalities. Specifically, IIE first learns sample-level representations by distilling modality-common information from the available modalities into learnable latent units. Then, it enhances modality-level representations by leveraging modality-specific information from other samples with the same modality, which is crucial for improving robustness in the presence of missing modalities. Finally, IIE ensures consistency between the enhanced modality-level and sample-level representations, combining the enhanced and initial representations to make predictions. Extensive experiments on three datasets demonstrate that the IIE framework significantly outperforms existing methods in terms of both effectiveness and robustness in handling MSA with missing modalities. Code is available at https://github.com/YetZzzzzz/IIE.
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 14066-14072
FAME: Fusion-Aware Multi-modal Ensemble for Social Media Popularity Prediction
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 14066-14072
Open AccessFAME: Fusion-Aware Multi-modal Ensemble for Social Media Popularity Prediction
- 22 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 9044-9058
Multi-Level Contrastive Learning for Multimodal Sentiment Analysis
Multimodal sentiment analysis has garnered increasing attention. The bulk of existing work in multimodal sentiment analysis primarily focuses on designing various networks to align and subsequently fuse representations from individual modalities. Contrastive learning, recognized for its intrinsic alignment capabilities, has also been extensively applied in multimodal sentiment analysis. However, current contrastive learning methods are often limited to pairwise modalities and typically perform contrastive learning prior to modality fusion, neglecting the consistency of interactions across multiple modalities. Moreover, they overlook the overall consistency within samples. To address these issues, we introduce a novel Multi-Level Contrastive Learning (MLCL) framework for multimodal sentiment analysis, composed of Uni-Modal Contrastive Learning (UMCL), Bi-Modal Contrastive Learning (BMCL) and Tri-Modal Contrastive Learning (TMCL). UMCL enhances intra-modal representations by creating positive pairs using modality-specific random dropout, while BMCL leverages the asymmetry of attention mechanisms, using two directional attentions as positive samples. TMCL aligns non-overlapping uni-modal and bi-modal representations, underscoring the complementarity of tri-modal information. The effectiveness of MLCL is demonstrated through its performance on multiple datasets. Our comprehensive experiments across multiple datasets demonstrate the superiority of the MLCL framework, which achieves new state-of-the-art performance.
- 22 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 9044-9058
Multi-Level Contrastive Learning for Multimodal Sentiment Analysis
Multimodal sentiment analysis has garnered increasing attention. The bulk of existing work in multimodal sentiment analysis primarily focuses on designing various networks to align and subsequently fuse representations from individual modalities. Contrastive learning, recognized for its intrinsic alignment capabilities, has also been extensively applied in multimodal sentiment analysis. However, current contrastive learning methods are often limited to pairwise modalities and typically perform contrastive learning prior to modality fusion, neglecting the consistency of interactions across multiple modalities. Moreover, they overlook the overall consistency within samples. To address these issues, we introduce a novel Multi-Level Contrastive Learning (MLCL) framework for multimodal sentiment analysis, composed of Uni-Modal Contrastive Learning (UMCL), Bi-Modal Contrastive Learning (BMCL) and Tri-Modal Contrastive Learning (TMCL). UMCL enhances intra-modal representations by creating positive pairs using modality-specific random dropout, while BMCL leverages the asymmetry of attention mechanisms, using two directional attentions as positive samples. TMCL aligns non-overlapping uni-modal and bi-modal representations, underscoring the complementarity of tri-modal information. The effectiveness of MLCL is demonstrated through its performance on multiple datasets. Our comprehensive experiments across multiple datasets demonstrate the superiority of the MLCL framework, which achieves new state-of-the-art performance.
- 21 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 16 (4) , 3329-3341
Enhanced Emotion Recognition in Conversations Through Hybrid Context Encoding and Latent Dependency Mining
Emotion recognition in conversations (ERC) is a pivotal component of affective computing, involving a common two-stage paradigm where pre-trained language models first extract context-independent features, followed by the encoding of contextual information and the modeling of emotional dependencies. This paradigm faces two challenges: (1) Existing methods struggle to capture both the intra-dialogue emotional continuity and the inter-dialogue semantic similarity. (2) The complexity of emotional elicitation processes gives rise to entangled dependencies, termed “latent dependencies”, which are difficult for current methods to detect and analyze. To overcome these challenges, we propose a Hybrid-Context Encoder with an Automated Latent Dependency Mining model for ERC. Specifically, we examine the emotional continuity and the semantic similarity from the standpoint of context encoders. We experimentally find that context encoders with different architectures exhibit distinct benefits. Based on these findings, we design a hybrid contextual encoding module that effectively combines the strengths of various encoders. Additionally, we design a lightweight generative module for latent dependency mining that autonomously generates a context mask, enabling the effective discovery of latent dependencies. We conduct extensive experiments on three datasets in the text modality. Our model achieves the best performance, which validates the superiority of our approach.
- 21 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 16 (4) , 3329-3341
Enhanced Emotion Recognition in Conversations Through Hybrid Context Encoding and Latent Dependency Mining
Emotion recognition in conversations (ERC) is a pivotal component of affective computing, involving a common two-stage paradigm where pre-trained language models first extract context-independent features, followed by the encoding of contextual information and the modeling of emotional dependencies. This paradigm faces two challenges: (1) Existing methods struggle to capture both the intra-dialogue emotional continuity and the inter-dialogue semantic similarity. (2) The complexity of emotional elicitation processes gives rise to entangled dependencies, termed “latent dependencies”, which are difficult for current methods to detect and analyze. To overcome these challenges, we propose a Hybrid-Context Encoder with an Automated Latent Dependency Mining model for ERC. Specifically, we examine the emotional continuity and the semantic similarity from the standpoint of context encoders. We experimentally find that context encoders with different architectures exhibit distinct benefits. Based on these findings, we design a hybrid contextual encoding module that effectively combines the strengths of various encoders. Additionally, we design a lightweight generative module for latent dependency mining that autonomously generates a context mask, enabling the effective discovery of latent dependencies. We conduct extensive experiments on three datasets in the text modality. Our model achieves the best performance, which validates the superiority of our approach.
- Open Access
- 22 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 4160-4170
ETS-MM: A Multi-Modal Social Bot Detection Model Based on Enhanced Textual Semantic Representation
- 22 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 4160-4170
Open AccessETS-MM: A Multi-Modal Social Bot Detection Model Based on Enhanced Textual Semantic Representation
- Open Access
- 1 January 2025
- Published by American Association for the Advancement of Science (AAAS) in Research
- Vol. 8, 0729
R3DG: Retrieve, Rank, and Reconstruction with Different Granularities for Multimodal Sentiment Analysis
Multimodal sentiment analysis (MSA) aims to assess emotional states by integrating information from text, audio, and video. However, the heterogeneous nature of these modalities presents substantial challenges for accurate sentiment prediction. Existing approaches typically align pairs of modalities using attention mechanisms or contrastive learning, which are computationally expensive. Additionally, they often rely on a single granularity of alignment, either by averaging features over all time steps or aligning features at each individual time step. These approaches overlook the fact that emotional expression can vary across individuals and contexts, requiring multiple granularities to capture emotion effectively. To address these challenges, we propose a novel framework, Retrieve, Rank, and Reconstruction with Different Granularities (R3DG). R3DG segments the audio and video modalities into multiple representations at varying granularities based on their temporal durations. It then selects the most relevant representations that align closely with the text modality. To preserve the original information, R3DG reconstructs the audio and video data using the selected representations. Finally, the fused audio, video, and text features are aligned and combined for sentiment prediction, reducing the need for multiple alignment steps. Extensive experiments on 5 benchmark MSA datasets demonstrate that R3DG outperforms existing methods and achieves substantial reductions in computational time. Code is available at https://github.com/YetZzzzzz/R3DG .
- 1 January 2025
- Published by American Association for the Advancement of Science (AAAS) in Research
- Vol. 8, 0729
Open AccessR3DG: Retrieve, Rank, and Reconstruction with Different Granularities for Multimodal Sentiment Analysis
Multimodal sentiment analysis (MSA) aims to assess emotional states by integrating information from text, audio, and video. However, the heterogeneous nature of these modalities presents substantial challenges for accurate sentiment prediction. Existing approaches typically align pairs of modalities using attention mechanisms or contrastive learning, which are computationally expensive. Additionally, they often rely on a single granularity of alignment, either by averaging features over all time steps or aligning features at each individual time step. These approaches overlook the fact that emotional expression can vary across individuals and contexts, requiring multiple granularities to capture emotion effectively. To address these challenges, we propose a novel framework, Retrieve, Rank, and Reconstruction with Different Granularities (R3DG). R3DG segments the audio and video modalities into multiple representations at varying granularities based on their temporal durations. It then selects the most relevant representations that align closely with the text modality. To preserve the original information, R3DG reconstructs the audio and video data using the selected representations. Finally, the fused audio, video, and text features are aligned and combined for sentiment prediction, reducing the need for multiple alignment steps. Extensive experiments on 5 benchmark MSA datasets demonstrate that R3DG outperforms existing methods and achieves substantial reductions in computational time. Code is available at https://github.com/YetZzzzzz/R3DG .
- 28 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 37 (2) , 739-753
Hierarchical Denoising for Robust Social Recommendation
Social recommendations leverage social networks to augment the performance of recommender systems. However, the critical task of denoising social information has not been thoroughly investigated in prior research. In this study, we introduce a hierarchical denoising robust social recommendation model to tackle noise at two levels: 1) intra-domain noise, resulting from user multi-faceted social trust relationships, and 2) inter-domain noise, stemming from the entanglement of the latent factors over heterogeneous relations (e.g., user-item interactions, user-user trust relationships). Specifically, our model advances a preference and social psychology-aware methodology for the fine-grained and multi-perspective estimation of tie strength within social networks. This serves as a precursor to an edge weight-guided edge pruning strategy that refines the model's diversity and robustness by dynamically filtering social ties. Additionally, we propose a user interest-aware cross-domain denoising gate, which not only filters noise during the knowledge transfer process but also captures the high-dimensional, nonlinear information prevalent in social domains. We conduct extensive experiments on three real-world datasets to validate the effectiveness of our proposed model against state-of-the-art baselines. We perform empirical studies on synthetic datasets to validate the strong robustness of our proposed model.
- 28 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 37 (2) , 739-753
Hierarchical Denoising for Robust Social Recommendation
Social recommendations leverage social networks to augment the performance of recommender systems. However, the critical task of denoising social information has not been thoroughly investigated in prior research. In this study, we introduce a hierarchical denoising robust social recommendation model to tackle noise at two levels: 1) intra-domain noise, resulting from user multi-faceted social trust relationships, and 2) inter-domain noise, stemming from the entanglement of the latent factors over heterogeneous relations (e.g., user-item interactions, user-user trust relationships). Specifically, our model advances a preference and social psychology-aware methodology for the fine-grained and multi-perspective estimation of tie strength within social networks. This serves as a precursor to an edge weight-guided edge pruning strategy that refines the model's diversity and robustness by dynamically filtering social ties. Additionally, we propose a user interest-aware cross-domain denoising gate, which not only filters noise during the knowledge transfer process but also captures the high-dimensional, nonlinear information prevalent in social domains. We conduct extensive experiments on three real-world datasets to validate the effectiveness of our proposed model against state-of-the-art baselines. We perform empirical studies on synthetic datasets to validate the strong robustness of our proposed model.