Chen Gong
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Chen Gong has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
Chuanliu Fan
2 shared publicationsGuohong Fu
2 shared publications- Guohong Fu
Institute of Artificial Intelligence, School of Computer Science and Technology, Soochow University, Suzhou, China
2 shared publications Junjie Wu
2 shared publications- Min Zhang
Harbin Institute of Technology
2 shared publications Yumeng Fu
2 shared publications
Latest Publications
- Open Access
- 22 April 2026
- Published by Elsevier in SSRN Electronic Journal
SDLE: Shallow-deep learning for out-of-context misinformation detection using multimodal large language models
Out-Of-Context (OOC) misinformation detection aims to identify whether a genuine image is repurposed in a misleading textual content. Such misinformation arises when genuine images are detached from their original contexts and paired with semantically related but false texts, making detection based solely on image-text pairs challenging. Existing methods primarily exploit internal knowledge from image-text pairs or external knowledge from retrieved evidence, yet they often fail to establish deep alignment between these sources due to limited interaction. To address these limitations, we propose SDLE, a novel Shallow-Deep Learning framework that enhances Multimodal Large Language Models (MLLMs) for OOC misinformation detection. Specifically, SDLE first retrieves external evidence from various sources to gather contextual information related to image-text pairs. To build a shallow relationship between internal and external knowledge, SDLE designs a prompt learning method to reduce noise and capture key evidence. Subsequently, to build a deep relationship between them, SDLE aligns MLLMs for evidence-based rationale generation. Furthermore, we adopt instruction tuning techniques to empower the model with both judgments and explanations in the realm of OOC misinformation detection. Comprehensive experiments on two widely used benchmark datasets, NewsCLIPpings and VERITE, show that SDLE consistently outperforms existing methods, achieving F1-score improvements of 1.4% and 2.0%, respectively.
- 22 April 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessSDLE: Shallow-deep learning for out-of-context misinformation detection using multimodal large language models
Out-Of-Context (OOC) misinformation detection aims to identify whether a genuine image is repurposed in a misleading textual content. Such misinformation arises when genuine images are detached from their original contexts and paired with semantically related but false texts, making detection based solely on image-text pairs challenging. Existing methods primarily exploit internal knowledge from image-text pairs or external knowledge from retrieved evidence, yet they often fail to establish deep alignment between these sources due to limited interaction. To address these limitations, we propose SDLE, a novel Shallow-Deep Learning framework that enhances Multimodal Large Language Models (MLLMs) for OOC misinformation detection. Specifically, SDLE first retrieves external evidence from various sources to gather contextual information related to image-text pairs. To build a shallow relationship between internal and external knowledge, SDLE designs a prompt learning method to reduce noise and capture key evidence. Subsequently, to build a deep relationship between them, SDLE aligns MLLMs for evidence-based rationale generation. Furthermore, we adopt instruction tuning techniques to empower the model with both judgments and explanations in the realm of OOC misinformation detection. Comprehensive experiments on two widely used benchmark datasets, NewsCLIPpings and VERITE, show that SDLE consistently outperforms existing methods, achieving F1-score improvements of 1.4% and 2.0%, respectively.
- Open Access
- 21 April 2026
- Published by Elsevier in SSRN Electronic Journal
SDLE: Shallow-deep learning for out-of-context misinformation detection using multimodal large language models
Out-Of-Context (OOC) misinformation detection aims to identify whether a genuine image is repurposed in a misleading textual content. Such misinformation arises when genuine images are detached from their original contexts and paired with semantically related but false texts, making detection based solely on image-text pairs challenging. Existing methods primarily exploit internal knowledge from image-text pairs or external knowledge from retrieved evidence, yet they often fail to establish deep alignment between these sources due to limited interaction. To address these limitations, we propose SDLE, a novel Shallow-Deep Learning framework that enhances Multimodal Large Language Models (MLLMs) for OOC misinformation detection. Specifically, SDLE first retrieves external evidence from various sources to gather contextual information related to image-text pairs. To build a shallow relationship between internal and external knowledge, SDLE designs a prompt learning method to reduce noise and capture key evidence. Subsequently, to build a deep relationship between them, SDLE aligns MLLMs for evidence-based rationale generation. Furthermore, we adopt instruction tuning techniques to empower the model with both judgments and explanations in the realm of OOC misinformation detection. Comprehensive experiments on two widely used benchmark datasets, NewsCLIPpings and VERITE, show that SDLE consistently outperforms existing methods, achieving F1-score improvements of 1.4% and 2.0%, respectively.
- 21 April 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessSDLE: Shallow-deep learning for out-of-context misinformation detection using multimodal large language models
Out-Of-Context (OOC) misinformation detection aims to identify whether a genuine image is repurposed in a misleading textual content. Such misinformation arises when genuine images are detached from their original contexts and paired with semantically related but false texts, making detection based solely on image-text pairs challenging. Existing methods primarily exploit internal knowledge from image-text pairs or external knowledge from retrieved evidence, yet they often fail to establish deep alignment between these sources due to limited interaction. To address these limitations, we propose SDLE, a novel Shallow-Deep Learning framework that enhances Multimodal Large Language Models (MLLMs) for OOC misinformation detection. Specifically, SDLE first retrieves external evidence from various sources to gather contextual information related to image-text pairs. To build a shallow relationship between internal and external knowledge, SDLE designs a prompt learning method to reduce noise and capture key evidence. Subsequently, to build a deep relationship between them, SDLE aligns MLLMs for evidence-based rationale generation. Furthermore, we adopt instruction tuning techniques to empower the model with both judgments and explanations in the realm of OOC misinformation detection. Comprehensive experiments on two widely used benchmark datasets, NewsCLIPpings and VERITE, show that SDLE consistently outperforms existing methods, achieving F1-score improvements of 1.4% and 2.0%, respectively.
- 4 October 2025
- Published by Elsevier in Information Processing & Management
- Vol. 63 (2)
Enhancing multi-modal aspect-based sentiment classification via emotional semantic-aware cross-modal relation inference
- 4 October 2025
- Published by Elsevier in Information Processing & Management
- Vol. 63 (2)
Enhancing multi-modal aspect-based sentiment classification via emotional semantic-aware cross-modal relation inference
- 31 May 2025
- Published by Elsevier in Expert Systems with Applications
- Vol. 290
Annotation error detection in painstakingly annotated data: Part-of-speech tagging as a case study
- 31 May 2025
- Published by Elsevier in Expert Systems with Applications
- Vol. 290
Annotation error detection in painstakingly annotated data: Part-of-speech tagging as a case study
- Open Access
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 3209-3218
MCG-MNER: A Multi-Granularity Cross-Modality Generative Framework for Multimodal NER with Instruction
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 3209-3218
Open AccessMCG-MNER: A Multi-Granularity Cross-Modality Generative Framework for Multimodal NER with Instruction
- 6 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 30 (23299290) , 1624-1636
Neural Coupled Sequence Labeling for Heterogeneous Annotation Conversion
Supervised statistical models rely on large-scale high-quality labeled data, which is important for model training but expensive to construct. Therefore, instead of constructing new dataset, researchers have attempted to make full use of various existing heterogeneous datasets to boost model performance, considering it is ubiquitous that the same task may have multiple annotated data following different and incompatible annotation guidelines. Representative methods include the guide-feature method which use the knowledge projected from the source-side to the target-side as extra features for target model guidance, and the multi-task learning (MTL) method which simultaneously train on multiple heterogeneous annotations with shared parameters to gain resource-share knowledge. Though effective, the guide-feature method fails to directly use the source-side data as training data, and the MTL method ignores the implicit mappings between heterogeneous datasets. Compared with the above methods, directly converting the heterogeneous datasets into homogeneous datasets for target model training is a more straightforward and effective way to fully exploit heterogeneous resources. In this work, we propose a neural coupled sequence labeling model for heterogeneous annotation conversion. First, for each token, we map a given one-side tag into a set of bundled tags by concatenating the tag with all the possible tags at the other side. Then, we build a neural coupled model over the bundled tag space. Finally, we convert heterogeneous annotations into homogeneous annotations by performing constraint decoding on the coupled model. We also propose a pruning strategy to address the oversize issue of the bundled tag space, which improves efficiency without hurting model performance. Experiments for part-of-speech (POS) tagging, word segmentation (WS), and WS\&POS tagging tasks show that our proposed neural coupled model consistently outperforms several benchmark models for all the three tasks by large margin.
- 6 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 30 (23299290) , 1624-1636
Neural Coupled Sequence Labeling for Heterogeneous Annotation Conversion
Supervised statistical models rely on large-scale high-quality labeled data, which is important for model training but expensive to construct. Therefore, instead of constructing new dataset, researchers have attempted to make full use of various existing heterogeneous datasets to boost model performance, considering it is ubiquitous that the same task may have multiple annotated data following different and incompatible annotation guidelines. Representative methods include the guide-feature method which use the knowledge projected from the source-side to the target-side as extra features for target model guidance, and the multi-task learning (MTL) method which simultaneously train on multiple heterogeneous annotations with shared parameters to gain resource-share knowledge. Though effective, the guide-feature method fails to directly use the source-side data as training data, and the MTL method ignores the implicit mappings between heterogeneous datasets. Compared with the above methods, directly converting the heterogeneous datasets into homogeneous datasets for target model training is a more straightforward and effective way to fully exploit heterogeneous resources. In this work, we propose a neural coupled sequence labeling model for heterogeneous annotation conversion. First, for each token, we map a given one-side tag into a set of bundled tags by concatenating the tag with all the possible tags at the other side. Then, we build a neural coupled model over the bundled tag space. Finally, we convert heterogeneous annotations into homogeneous annotations by performing constraint decoding on the coupled model. We also propose a pruning strategy to address the oversize issue of the bundled tag space, which improves efficiency without hurting model performance. Experiments for part-of-speech (POS) tagging, word segmentation (WS), and WS\&POS tagging tasks show that our proposed neural coupled model consistently outperforms several benchmark models for all the three tasks by large margin.