Zhiyuan Wen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Zhiyuan Wen has a total of 53 co-authors. Here are the top co-authors by number of shared publications:
- Ruifeng Xu
Harbin Institute of Technology, Shenzhen, China
9 shared publications - Qianlong Wang
Harbin Institute of Technology, Shenzhen, China
6 shared publications - Bin Liang
The Chinese University of Hong Kong, Hong Kong, China
5 shared publications - Keyang Ding
Harbin Institute of Technology Shenzhen China
3 shared publications Qianlong Wang
3 shared publications- Jing Li
The Hong Kong Polytechnic University, Hong Kong, China
2 shared publications
Latest Publications
- Open Access
- 27 June 2026
- Published by Elsevier in SSRN Electronic Journal
Scene Graph Enhanced Multi-Perspective Gated Interaction Framework for Multimodal Sarcasm Detection
Sarcasm, a complex linguistic phenomenon where literal and intended meanings differ, is widespread on social media. Effective sarcasm detection is vital for interpreting user intent and behavior accurately. Multimodal sarcasm detection aims to identify sarcastic expressions from given multimodal data, primarily images and text. Although various effective methods have been proposed, especially graph-based approaches, they often fail to thoroughly explore intra-modal inconsistencies, particularly within the image modality, thereby limiting detection performance. To address this limitation, we introduce scene graph knowledge, a directed graph that clearly depicts intra-modal semantic relationships. Based on this, we propose a Scene Graph Enhanced Multi-Perspective Gated Interaction Framework for multimodal sarcasm detection. This method first constructs visual and textual relationship graphs, incorporating semantic relationships as nodes to enhance the model's focus on intra-modal relationship modeling. The two graphs are subsequently fused, and cross-modal interaction nodes are added to form a cross-modal interaction graph. We then employ a multi-perspective gated interaction mechanism, centering on text, image, visual relationship graph, and textual relationship graph, to aggregate useful information from adjacent modalities. This comprehensive approach provides a deep and nuanced understanding of sarcastic expressions. Experimental results demonstrate that our proposed method effectively captures inconsistencies within and between modalities, achieving superior performance compared to existing state-of-the-art methods.
- 27 June 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessScene Graph Enhanced Multi-Perspective Gated Interaction Framework for Multimodal Sarcasm Detection
Sarcasm, a complex linguistic phenomenon where literal and intended meanings differ, is widespread on social media. Effective sarcasm detection is vital for interpreting user intent and behavior accurately. Multimodal sarcasm detection aims to identify sarcastic expressions from given multimodal data, primarily images and text. Although various effective methods have been proposed, especially graph-based approaches, they often fail to thoroughly explore intra-modal inconsistencies, particularly within the image modality, thereby limiting detection performance. To address this limitation, we introduce scene graph knowledge, a directed graph that clearly depicts intra-modal semantic relationships. Based on this, we propose a Scene Graph Enhanced Multi-Perspective Gated Interaction Framework for multimodal sarcasm detection. This method first constructs visual and textual relationship graphs, incorporating semantic relationships as nodes to enhance the model's focus on intra-modal relationship modeling. The two graphs are subsequently fused, and cross-modal interaction nodes are added to form a cross-modal interaction graph. We then employ a multi-perspective gated interaction mechanism, centering on text, image, visual relationship graph, and textual relationship graph, to aggregate useful information from adjacent modalities. This comprehensive approach provides a deep and nuanced understanding of sarcastic expressions. Experimental results demonstrate that our proposed method effectively captures inconsistencies within and between modalities, achieving superior performance compared to existing state-of-the-art methods.
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 64, 5528915
Sparse Tensor Attention for Hyperspectral Unmixing
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 64, 5528915
Sparse Tensor Attention for Hyperspectral Unmixing
- 29 December 2025
- Published by Elsevier in Information Fusion
- Vol. 130
Internet meme on social media: A comprehensive review and new perspectives
- 29 December 2025
- Published by Elsevier in Information Fusion
- Vol. 130
Internet meme on social media: A comprehensive review and new perspectives
- 24 April 2025
- Published by Elsevier in Information Processing & Management
- Vol. 62 (5)
FGVIrony: A Chinese Dataset of Fine-grained Verbal Irony
- 24 April 2025
- Published by Elsevier in Information Processing & Management
- Vol. 62 (5)
FGVIrony: A Chinese Dataset of Fine-grained Verbal Irony
- 22 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 16 (1) , 264-276
Cross-Domain Sentiment Analysis via Disentangled Representation and Prototypical Learning
Cross-domain sentiment analysis (CDSA) aims to predict the sentiment polarities of reviews in the target domain using a sentiment classifier learned from the source labeled domain. Most existing studies are dominant with adversarial learning methods and focus on learning domain-invariant sentiment representations in both the source and target domains. However, since sentiment-specific features are not explicitly decoupled, the model may confuse domain features with sentiment features, thus affecting its generalization ability on target domains. Unlike previous studies, in this paper, we tackle the CDSA task from the view of disentangled representation learning, which explicitly learns the disentangled representations of review, focusing in particular on sentiment and domain semantics. Specifically, we disentangle sentiment-specific and domain-specific features from the text representation of the review by two different linear transformations. Then, we introduce a straightforward disentangled loss to disallow the sentiment-specific feature to capture domain information. Moreover, we leverage target unlabeled data to improve the quality of the learned sentiment-specific features via prototypical learning. It indirectly encourages the sentiment-specific features of target samples having potentially different classes more discriminative. Extensive experiments on widely used CDSA datasets show that our method surpasses competitive baselines and achieves new state-of-the-art results, demonstrating its effectiveness and superiority.
- 22 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 16 (1) , 264-276
Cross-Domain Sentiment Analysis via Disentangled Representation and Prototypical Learning
Cross-domain sentiment analysis (CDSA) aims to predict the sentiment polarities of reviews in the target domain using a sentiment classifier learned from the source labeled domain. Most existing studies are dominant with adversarial learning methods and focus on learning domain-invariant sentiment representations in both the source and target domains. However, since sentiment-specific features are not explicitly decoupled, the model may confuse domain features with sentiment features, thus affecting its generalization ability on target domains. Unlike previous studies, in this paper, we tackle the CDSA task from the view of disentangled representation learning, which explicitly learns the disentangled representations of review, focusing in particular on sentiment and domain semantics. Specifically, we disentangle sentiment-specific and domain-specific features from the text representation of the review by two different linear transformations. Then, we introduce a straightforward disentangled loss to disallow the sentiment-specific feature to capture domain information. Moreover, we leverage target unlabeled data to improve the quality of the learned sentiment-specific features via prototypical learning. It indirectly encourages the sentiment-specific features of target samples having potentially different classes more discriminative. Extensive experiments on widely used CDSA datasets show that our method surpasses competitive baselines and achieves new state-of-the-art results, demonstrating its effectiveness and superiority.
- 2 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Social Systems
- Vol. 11 (3) , 3362-3375
LCSEP: A Large-Scale Chinese Dataset for Social Emotion Prediction to Online Trending Topics
In this article, we present our work in social emotion prediction to online trending topics. While most prior works focus on emotion from writers or the readers’ emotions evoked by news articles, we investigate discussions from massive social media users and explore the public feelings to the online trending topic. We employ user-generated “#hashtags” to indicate online trending topics and construct a large-scale Chinese dataset for social emotion prediction (LCSEP) to trending topics collected from the Chinese microblog Sina Weibo. It contains more than 20 000 trending topics, each with social emotions voted in 24 fine-grained types, and gathers hashtags, posts, comments, and related metadata to give each trending topic a thorough context. We also propose a Hashtag- and Topic-Enhanced Attention Model (HTEAM) that combines a pretrained BERT model, a neural topic model, and an attention mechanism via joint training to understand social emotion. Experiments show that HTEAM outperforms baselines and achieves the state-of-the-art result.
- 2 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Social Systems
- Vol. 11 (3) , 3362-3375
LCSEP: A Large-Scale Chinese Dataset for Social Emotion Prediction to Online Trending Topics
In this article, we present our work in social emotion prediction to online trending topics. While most prior works focus on emotion from writers or the readers’ emotions evoked by news articles, we investigate discussions from massive social media users and explore the public feelings to the online trending topic. We employ user-generated “#hashtags” to indicate online trending topics and construct a large-scale Chinese dataset for social emotion prediction (LCSEP) to trending topics collected from the Chinese microblog Sina Weibo. It contains more than 20 000 trending topics, each with social emotions voted in 24 fine-grained types, and gathers hashtags, posts, comments, and related metadata to give each trending topic a thorough context. We also propose a Hashtag- and Topic-Enhanced Attention Model (HTEAM) that combines a pretrained BERT model, a neural topic model, and an attention mechanism via joint training to understand social emotion. Experiments show that HTEAM outperforms baselines and achieves the state-of-the-art result.
- 15 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 15 (3) , 1264-1278
Image-to-Text Conversion and Aspect-Oriented Filtration for Multimodal Aspect-Based Sentiment Analysis
Multimodal aspect-based sentiment analysis (MABSA) aims to determine the sentiment polarity of each aspect mentioned in the text based on multimodal content. Various approaches have been proposed to model multimodal sentiment features for each aspect via modal interactions. However, most existing approaches have two shortcomings: (1) The representation gap between textual and visual modalities may increase the risk of misalignment in modal interactions; (2) In some examples where the image is not related to the text, the visual information may not enrich the textual modality when learning aspect-based sentiment features. In such cases, blindly leveraging information from visual modal may introduce noises in reasoning the aspect-based sentiment expressions. To tackle the shortcomings mentioned above, we propose an end-to-end MABSA framework with image conversion and noise filtration. Specifically, to bridge the representation gap in different modalities, we attempt to translate images into the input space of a pre-trained language model (PLM). To this end, we develop an image-to-text conversion module that can convert an image to an implicit sequence of token embedding. Moreover, an aspect-oriented filtration module is devised to alleviate the noise in the implicit token embeddings, which consists of two attention operations. The former aims to create an enhanced aspect embedding as a query, and the latter seeks to use this query to retrieve relevant auxiliary information from the implicit token embeddings to supplement the textual content. After filtering the noise, we leverage a PLM to encode the text, aspect, and image prompt derived from filtered implicit token embeddings as sentiment features to perform aspect-based sentiment prediction. Experimental results on two MABSA datasets show that our framework achieves state-of-the-art performance. Furthermore, extensive experimental analysis demonstrates the proposed framework has superior robustness and efficiency.
- 15 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 15 (3) , 1264-1278
Image-to-Text Conversion and Aspect-Oriented Filtration for Multimodal Aspect-Based Sentiment Analysis
Multimodal aspect-based sentiment analysis (MABSA) aims to determine the sentiment polarity of each aspect mentioned in the text based on multimodal content. Various approaches have been proposed to model multimodal sentiment features for each aspect via modal interactions. However, most existing approaches have two shortcomings: (1) The representation gap between textual and visual modalities may increase the risk of misalignment in modal interactions; (2) In some examples where the image is not related to the text, the visual information may not enrich the textual modality when learning aspect-based sentiment features. In such cases, blindly leveraging information from visual modal may introduce noises in reasoning the aspect-based sentiment expressions. To tackle the shortcomings mentioned above, we propose an end-to-end MABSA framework with image conversion and noise filtration. Specifically, to bridge the representation gap in different modalities, we attempt to translate images into the input space of a pre-trained language model (PLM). To this end, we develop an image-to-text conversion module that can convert an image to an implicit sequence of token embedding. Moreover, an aspect-oriented filtration module is devised to alleviate the noise in the implicit token embeddings, which consists of two attention operations. The former aims to create an enhanced aspect embedding as a query, and the latter seeks to use this query to retrieve relevant auxiliary information from the implicit token embeddings to supplement the textual content. After filtering the noise, we leverage a PLM to encode the text, aspect, and image prompt derived from filtered implicit token embeddings as sentiment features to perform aspect-based sentiment prediction. Experimental results on two MABSA datasets show that our framework achieves state-of-the-art performance. Furthermore, extensive experimental analysis demonstrates the proposed framework has superior robustness and efficiency.
- 28 June 2023
- Published by Springer Nature in International Journal of Machine Learning and Cybernetics
- Vol. 14 (12) , 4119-4134
Multi-perspective contrastive learning framework guided by sememe knowledge and label information for sarcasm detection
- 28 June 2023
- Published by Springer Nature in International Journal of Machine Learning and Cybernetics
- Vol. 14 (12) , 4119-4134
Multi-perspective contrastive learning framework guided by sememe knowledge and label information for sarcasm detection
- Open Access
- 14 June 2023
- Published by MDPI AG in Electronics
- Vol. 12 (12) , 2673
RDVI: A Retrieval–Detection Framework for Verbal Irony Detection
Verbal irony is a common form of expression used in daily communication, where the intended meaning is often opposite to the literal meaning. Accurately recognizing verbal irony is essential for any NLP application for which the understanding of the true user intentions is key to performing the underlying tasks. While existing research has made progress in this area, verbal irony often involves connotative knowledge that cannot be directly inferred from the text or its context, which limits the detection model’s ability to recognize and comprehend verbal irony. To address this issue, we propose a Retrieval–Detection method for Verbal Irony (RDVI). This approach improves the detection model’s ability to recognize and comprehend verbal irony by retrieving the connotative knowledge from the open domain and incorporating it into the model using prompt learning. The experimental results demonstrate that our proposed method outperforms state-of-the-art models.
- 14 June 2023
- Published by MDPI AG in Electronics
- Vol. 12 (12) , 2673
Open AccessRDVI: A Retrieval–Detection Framework for Verbal Irony Detection
Verbal irony is a common form of expression used in daily communication, where the intended meaning is often opposite to the literal meaning. Accurately recognizing verbal irony is essential for any NLP application for which the understanding of the true user intentions is key to performing the underlying tasks. While existing research has made progress in this area, verbal irony often involves connotative knowledge that cannot be directly inferred from the text or its context, which limits the detection model’s ability to recognize and comprehend verbal irony. To address this issue, we propose a Retrieval–Detection method for Verbal Irony (RDVI). This approach improves the detection model’s ability to recognize and comprehend verbal irony by retrieving the connotative knowledge from the open domain and incorporating it into the model using prompt learning. The experimental results demonstrate that our proposed method outperforms state-of-the-art models.
- 17 December 2022
- Published by Springer Nature
- p. 298-309
Interactive Fusion Network with Recurrent Attention for Multimodal Aspect-based Sentiment Analysis
- 17 December 2022
- Published by Springer Nature
- p. 298-309
Interactive Fusion Network with Recurrent Attention for Multimodal Aspect-based Sentiment Analysis