CGChen Gong

Chen Gong

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Chen Gong has a total of 13 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    SDLE: Shallow-deep learning for out-of-context misinformation detection using multimodal large language models

    by Junjie Wu, Yumeng Fu, Chuanliu Fan, Chen Gong, Guohong Fu

    Out-Of-Context (OOC) misinformation detection aims to identify whether a genuine image is repurposed in a misleading textual content. Such misinformation arises when genuine images are detached from their original contexts and paired with semantically related but false texts, making detection based solely on image-text pairs challenging. Existing methods primarily exploit internal knowledge from image-text pairs or external knowledge from retrieved evidence, yet they often fail to establish deep alignment between these sources due to limited interaction. To address these limitations, we propose SDLE, a novel Shallow-Deep Learning framework that enhances Multimodal Large Language Models (MLLMs) for OOC misinformation detection. Specifically, SDLE first retrieves external evidence from various sources to gather contextual information related to image-text pairs. To build a shallow relationship between internal and external knowledge, SDLE designs a prompt learning method to reduce noise and capture key evidence. Subsequently, to build a deep relationship between them, SDLE aligns MLLMs for evidence-based rationale generation. Furthermore, we adopt instruction tuning techniques to empower the model with both judgments and explanations in the realm of OOC misinformation detection. Comprehensive experiments on two widely used benchmark datasets, NewsCLIPpings and VERITE, show that SDLE consistently outperforms existing methods, achieving F1-score improvements of 1.4% and 2.0%, respectively.

  • Open Access

    SDLE: Shallow-deep learning for out-of-context misinformation detection using multimodal large language models

    by Junjie Wu, Yumeng Fu, Chuanliu Fan, Chen Gong, Guohong Fu

    Out-Of-Context (OOC) misinformation detection aims to identify whether a genuine image is repurposed in a misleading textual content. Such misinformation arises when genuine images are detached from their original contexts and paired with semantically related but false texts, making detection based solely on image-text pairs challenging. Existing methods primarily exploit internal knowledge from image-text pairs or external knowledge from retrieved evidence, yet they often fail to establish deep alignment between these sources due to limited interaction. To address these limitations, we propose SDLE, a novel Shallow-Deep Learning framework that enhances Multimodal Large Language Models (MLLMs) for OOC misinformation detection. Specifically, SDLE first retrieves external evidence from various sources to gather contextual information related to image-text pairs. To build a shallow relationship between internal and external knowledge, SDLE designs a prompt learning method to reduce noise and capture key evidence. Subsequently, to build a deep relationship between them, SDLE aligns MLLMs for evidence-based rationale generation. Furthermore, we adopt instruction tuning techniques to empower the model with both judgments and explanations in the realm of OOC misinformation detection. Comprehensive experiments on two widely used benchmark datasets, NewsCLIPpings and VERITE, show that SDLE consistently outperforms existing methods, achieving F1-score improvements of 1.4% and 2.0%, respectively.

  • Neural Coupled Sequence Labeling for Heterogeneous Annotation Conversion

    by Chen Gong, Zhenghua Li, Min Zhang

    Supervised statistical models rely on large-scale high-quality labeled data, which is important for model training but expensive to construct. Therefore, instead of constructing new dataset, researchers have attempted to make full use of various existing heterogeneous datasets to boost model performance, considering it is ubiquitous that the same task may have multiple annotated data following different and incompatible annotation guidelines. Representative methods include the guide-feature method which use the knowledge projected from the source-side to the target-side as extra features for target model guidance, and the multi-task learning (MTL) method which simultaneously train on multiple heterogeneous annotations with shared parameters to gain resource-share knowledge. Though effective, the guide-feature method fails to directly use the source-side data as training data, and the MTL method ignores the implicit mappings between heterogeneous datasets. Compared with the above methods, directly converting the heterogeneous datasets into homogeneous datasets for target model training is a more straightforward and effective way to fully exploit heterogeneous resources. In this work, we propose a neural coupled sequence labeling model for heterogeneous annotation conversion. First, for each token, we map a given one-side tag into a set of bundled tags by concatenating the tag with all the possible tags at the other side. Then, we build a neural coupled model over the bundled tag space. Finally, we convert heterogeneous annotations into homogeneous annotations by performing constraint decoding on the coupled model. We also propose a pruning strategy to address the oversize issue of the bundled tag space, which improves efficiency without hurting model performance. Experiments for part-of-speech (POS) tagging, word segmentation (WS), and WS\&POS tagging tasks show that our proposed neural coupled model consistently outperforms several benchmark models for all the three tasks by large margin.