Yongmei Zhou

The prevalence of fake news online has become a significant societal concern. To combat this, multimodal detection techniques based on images and text have shown promise. Yet, these methods struggle to analyze complex relationships within and between modalities due to the diverse discriminative elements in the news content. In addition, research on multimodal and multi-class fake news detection remains insufficient. To address the above challenges, in this article, we propose a novel detection model, GS ² F, leveraging g raph s tructure and g uided s emantic f usion. Specifically, we construct a multimodal graph structure to align two modalities and employ graph contrastive learning for refined fusion representations. Furthermore, a guided semantic fusion module is introduced to maximize the utilization of single-modal information and a dynamic contribution assignment layer is designed to weigh the importance of image, text, and multimodal features. Experimental results on Fakeddit demonstrate that our model outperforms existing methods, marking a step forward in the multimodal and multi-class fake news detection.

Weixiong Zheng, Peijian Zeng, YiWei Li, Hongyan Wu, Nankai Lin, Junhao Chen, Aimin Yang, Yongmei Zhou. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Large language models (LLMs) excel in various tasks but remain vulnerable to jailbreak attacks, where adversaries manipulate prompts to generate harmful outputs. Examining jailbreak prompts helps uncover the shortcomings of LLMs. However, current jailbreak methods and the target model's defenses are engaged in an independent and adversarial process, resulting in the need for frequent attack iterations and redesigning attacks for different models. To address these gaps, we propose a Reverse Embedded Defense Attack (REDA) mechanism that disguises the attack intention as the "defense". intention against harmful content. Specifically, REDA starts from the target response, guiding the model to embed harmful content within its defensive measures, thereby relegating harmful content to a secondary role and making the model believe it is performing a defensive task. The attacking model considers that it is guiding the target model to deal with harmful content, while the target model thinks it is performing a defensive task, creating an illusion of cooperation between the two. Additionally, to enhance the model's confidence and guidance in "defensive" intentions, we adopt in-context learning (ICL) with a small number of attack examples and construct a corresponding dataset of attack examples. Extensive evaluations demonstrate that the REDA method enables cross-model attacks without the need to redesign attack strategies for different models, enables successful jailbreak in one iteration, and outperforms existing methods on both open-source and closed-source models.

The Aspect Sentiment Triplet Extraction (ASTE) task, which has the goal of extracting sentiment triplets from sentences, has been solved by the Machine Reading Comprehension (MRC) framework. Although the MRC model can effectively handle the ASTE task, there still remain issues. Traditional attention models do not explicitly prioritize and concentrate on the significance of words, which can result in the incorrect emphasis on less crucial words and assign higher attention weights to less important terms. Therefore, in this paper, a syntactically guided model is proposed. It incorporates syntax relationships from input to constrain the attention mechanism. We developed a model called Syntax-Guided Muti-Turn Machine Reading Comprehension (SG-MTMRC), which incorporates syntax relationships into the layer of self-attention by leveraging a Syntax-Guided Network (SG-NET). It creates a syntax-guided self-attention layer to enhance the input representation. Then, we put them into a Muti-Turn Machine Reading Comprehension (MTMRC) model for further processing. Four benchmark datasets are being used for extensive experiments so as to validate our approach's effectiveness. The experiment results indicate significant performance improvement achieved by the SG-MTMRC model.

The task of retrieving audio content relevant to lyric queries and vice versa plays a critical role in music-oriented applications. In this process, robust feature representations have to be learned for two modalities. Furthermore, interactions between different modalities should be properly captured at a fine-grained level. Existing approaches can effectively extract modal representations and perform retrieving between different modalities through alignment. However, these approaches model interactions between audio and lyrics in a coarse-grained manner. Especially the input features and interactions between enhanced representations produced by the alignment module are largely ignored, resulting in low-quality modality representations for final retrieval. This paper presents a novel method named CMRF that accomplishes cross-modal interactions via a reinforcement feedback procedure to learn high-quality multi-modal embeddings. Initially, we implicitly assimilate representations across distinct modalities via directional pairwise cross-modal attention. Subsequently, our approach recurrently identifies pivotal constituents within these elevated-level attributes to engage with the primary input features via reinforcement learning, thus augmenting the quality of multi-modal embeddings. In addition, we introduce a novel audio-lyrics dataset AL-song , which consists of paired audio with corresponding lyrics for the audio-lyrics retrieval task. The empirical findings derived from the AL-song dataset and the benchmark dataset Sounddescs substantiate the efficacy and efficiency of CMRF when juxtaposed with state-of-the-art methodologies.